{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Maximum Entropy Model and Expectation-maximization algorithm，最大熵模型与EM算法。\n",
    "\n",
    "## 阅读指南\n",
    "\n",
    "1. 在线观看请使用Chrome浏览器，并安装插件：[MathJax Plugin for Github(需科学上网)](https://chrome.google.com/webstore/detail/mathjax-plugin-for-github/ioemnmodlmafdkllaclgeombjnmnbima)， 插件[Github地址](https://github.com/orsharir/github-mathjax)\n",
    "2. 或下载内容到本地，使用markdown相关软件打开，如：[Typora](https://typora.io/)\n",
    "3. **若数学公式显示出现问题大家也可通过jupyter notebook链接查看：[Maximum-Entropy-Model-and-Expectation-maximization-algorithm](https://nbviewer.jupyter.org/github/Knowledge-Precipitation-Tribe/Maximum-Entropy-Model-and-Expectation-maximization-algorithm/blob/master/jupyter%20notebook/Maximum-Entropy-Model-and-Expectation-maximization-algorithm.ipynb)**\n",
    "\n",
    "## Content\n",
    "\n",
    "- <a href = \"#熵等相关概念\">熵等相关概念</a>\n",
    "  - <a href = \"#信息量\">信息量</a>\n",
    "  - <a href = \"#熵\">熵</a>\n",
    "    - <a href = \"#联合熵\">联合熵</a>\n",
    "    - <a href = \"#条件熵\">条件熵</a>\n",
    "    - <a href = \"#互信息\">互信息</a>\n",
    "  - <a href = \"#相对熵\">相对熵</a>\n",
    "  - <a href = \"#交叉熵\">交叉熵</a>\n",
    "- <a href = \"#最大熵模型\">最大熵模型</a>\n",
    "- <a href = \"#Jensen不等式\">Jensen不等式</a>\n",
    "- <a href = \"#极大似然估计\">极大似然估计</a>\n",
    "- <a href = \"#极大似然估计与EM算法的适用问题\">极大似然估计与EM算法的适用问题</a>\n",
    "- <a href = \"#EM算法\">EM算法</a>\n",
    "  - <a href = \"#什么是EM算法\">什么是EM算法</a>\n",
    "  - <a href = \"#EM算法的通俗解释\">EM算法的通俗解释</a>\n",
    "    - <a href = \"#例子1\">例子1</a>\n",
    "    - <a href = \"#例子2\">例子2</a>\n",
    "  - <a href = \"#数学推导\">数学推导</a>\n",
    "  - <a href = \"#EM算法流程\">EM算法流程</a>\n",
    "  - <a href = \"#代码实现\">代码实现</a>\n",
    "    - <a href = \"#二维高斯分布\">二维高斯分布</a>\n",
    "    - <a href = \"#数据可视化\">数据可视化</a>\n",
    "    - <a href = \"#变量初始化\">变量初始化</a>\n",
    "    - <a href = \"#E步骤\">E步骤</a>\n",
    "    - <a href = \"#M步骤\">M步骤</a>\n",
    "    - <a href = \"#迭代求解\">迭代求解</a>\n",
    "    - <a href = \"#完整代码\">完整代码</a>\n",
    "- <a href = \"#参考文献\">参考文献</a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [Jensen不等式](#content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [极大似然估计](#content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [极大似然估计与EM算法的适用问题](#content)\n",
    "\n",
    "参考自详解EM算法与混合高斯模型$^{[6]}$。\n",
    "\n",
    "**极大似然估计：**\n",
    "\n",
    "适用于某组数据服从某一分布，我们现在得到了这组数据中的采样值，现在我们想求解这一分布的参数情况，我们可以采用极大似然的方法进行求解。就像之前介绍到的，在极大似然估计中我们是对似然函数求偏导，令其得0，然后得到模型的参数，但是如果数据是来自多个分布，就没办法通过求导得0的方法来得到参数的估计值了，这时就要用到EM算法。\n",
    "\n",
    "**EM算法：**\n",
    "\n",
    "适用于某组数据服从一个混合的分布(如果这几个分布都是高斯分布的话，那么这个就叫做高斯混合模型)。现在我们得到了这一混合分布中的一些采样数据，反推这些采样数据分别是属于哪一类的，这时候我们可以采用EM算法进行求解。\n",
    "\n",
    "![em-mle](../img/em-mle.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# [EM算法](#content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [什么是EM算法](#content)\n",
    "\n",
    "EM算法被称为数据挖掘的十大算法之一，在机器学习和数据挖掘领域占有很重要的地位。接下来我就用尽量通俗的语言来梳理一下EM算法。EM算法也称为期望最大化(Expectation-Maximum)算法，在很多地方都能看到他的身影，例如HMM以及LDA等。\n",
    "\n",
    "> 维基百科定义：\n",
    ">\n",
    "> **最大期望算法**（**Expectation-maximization algorithm**，又译**期望最大化算法**）在统计中被用于寻找，依赖于不可观察的隐性变量的概率模型中，参数的最大似然估计。\n",
    ">\n",
    "> 在[统计](https://zh.wikipedia.org/wiki/统计)[计算](https://zh.wikipedia.org/wiki/计算)中，**最大期望（EM）算法**是在[概率模型](https://zh.wikipedia.org/wiki/概率模型)中寻找[参数](https://zh.wikipedia.org/wiki/参数)[最大似然估计](https://zh.wikipedia.org/wiki/最大似然估计)或者[最大后验估计](https://zh.wikipedia.org/wiki/最大后验概率)的[算法](https://zh.wikipedia.org/wiki/算法)，其中概率模型依赖于无法观测的[隐变量](https://zh.wikipedia.org/wiki/隐变量)。最大期望算法经常用在[机器学习](https://zh.wikipedia.org/wiki/机器学习)和[计算机视觉](https://zh.wikipedia.org/wiki/计算机视觉)的[数据聚类](https://zh.wikipedia.org/wiki/数据聚类)（Data Clustering）领域。最大期望算法经过两个步骤交替进行计算，第一步是计算期望（E），利用对隐藏变量的现有估计值，计算其最大似然估计值；第二步是最大化（M），最大化在E步上求得的[最大似然值](https://zh.wikipedia.org/wiki/最大似然估计)来计算参数的值。M步上找到的参数估计值被用于下一个E步计算中，这个过程不断交替进行。\n",
    "\n",
    "看完是不是有种黑人问号脸的感觉。\n",
    "\n",
    "![heirenwenhao](../img/heirenwenhao.jpg)\n",
    "\n",
    "但是不要慌，我们接下来就来梳理一下这个EM算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [EM算法通俗解释](#content)\n",
    "\n",
    "在一些资料中首先会介绍一下什么是似然函数，什么是极大似然估计等等，我们这里先试着抛开这些东西不谈，直接来理解一下EM算法。\n",
    "\n",
    "**EM算法用最简单的话来说就是：给你一堆数据，然后告诉你这些数据是满足一个分布的，现在你就来推导一下参数是什么样的分布可以拟合这堆数据。**\n",
    "\n",
    "如果你了解过参数估计，那你可能会有一点头绪，就算没有头绪也没关系，现在我们来看一个例子。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [例子1](#content)\n",
    "\n",
    "例子可能并不那么严谨，欢迎批评指正。\n",
    "\n",
    "现在我们有4位志愿者，我们通过测量获得了他们的身高数据。但是这些志愿者当中有男性有女性，具体比例我们没有记录，所以不知道男女各有多少，假设男女各占的比例分别用$\\pi_{男},\\pi_{女}$表示。假设男性和女性的身高分别服从$\\mathrm{N}\\left(\\mu_{男}, \\sigma_{男}^{2}\\right)$和$\\mathrm{N}\\left(\\mu_{女}, \\sigma_{女}^{2}\\right)$的高斯分布，现在我们来估计一下这六个参数$\\mu_{男}, \\sigma_{男} ^ {2},\\mu_{女}, \\sigma_{女} ^ {2},\\pi_{男},\\pi_{女}$可能的值是多少。\n",
    "\n",
    "解：\n",
    "\n",
    "反正现在是让我们估计这几个参数，我们不妨瞎猜一下，就假设$\\mu_{男}=170, \\sigma_{男} ^ 2=36,\\mu_{女}=160, \\sigma_{女} ^ 2=25$，而且男女各一半我们用$\\pi_{男}=\\pi_{女}=0.5$表示。\n",
    "\n",
    "而且已知其是满足高斯分布的，那么我们可以得到高斯分布的概率密度函数：\n",
    "$$\n",
    "f(x)=\\frac{1}{\\sigma \\sqrt{2 \\pi}} e^{-\\frac{(x-\\mu)^{2}}{2 \\sigma^{2}}}\n",
    "$$\n",
    "现在我们拿到第一个志愿者的身高数据：$x_1=188$，我们现在来判断一下第一个志愿者属于男性的概率有多大。我们现在将188代入到概率密度函数当中\n",
    "$$\n",
    "\\begin{aligned}f(188 | \\mu_{男}, \\sigma_{男}^{2}) &=\\frac{1}{6 \\times \\sqrt{2 \\pi}} e^{-\\frac{(188-170)^{2}}{2 \\times 6 ^ 2}} \\\\&= 7.368 \\times 10^{-4} \\\\f(188 | \\mu_{女}, \\sigma_{女}^{2}) &=\\frac{1}{5 \\times \\sqrt{2 \\pi}} e^{-\\frac{(188-160)^{2}}{2 \\times 5 ^{2}}} \\\\&= 1.236 \\times 10^{-8}\\end{aligned}\n",
    "$$\n",
    "而且我们假设$\\pi_{男}=\\pi_{女}=0.5$，那么当前第一个志愿者为男性的概率\n",
    "$$\n",
    "\\frac{7.368 \\times 10^{-4} \\times 0.5}{7.368 \\times 10^{-4} \\times 0.5+1.236 \\times 10^{-8} \\times 0.5}=0.999\n",
    "$$\n",
    "为女性的概率\n",
    "$$\n",
    "\\frac{1.236 \\times 10^{-8} \\times 0.5}{7.368 \\times 10^{-4} \\times 0.5+1.236 \\times 10^{-8} \\times 0.5}=0.001\n",
    "$$\n",
    "看来我们猜的还可以，188属于男性的概率确实很高。Anyway，我们继续来算一下剩下的三个志愿者的数据，与第一个志愿的算法类似，这里就不再赘述了。\n",
    "\n",
    "剩下的三个志愿者的身高数据分别为：$x_2=158, x_3=165, x_4=170$。那我们就可以得到这样一组数据。\n",
    "$$\n",
    "x_{1} = 188 = \\left\\{\\begin{array}{ll}0.999, \\text {男} \\\\0.001, \\text {女}\\end{array}\\right. ,x_{2}= 158 = \\left\\{\\begin{array}{ll}0.098, \\text {男} \\\\0.901, \\text {女}\\end{array}\\right. ,x_{3} = 165 = \\left\\{\\begin{array}{ll}0.490, \\text {男} \\\\0.510, \\text {女}\\end{array}\\right. ,x_{4} = 170 = \\left\\{\\begin{array}{ll}0.529, \\text {男} \\\\0.471, \\text {女}\\end{array}\\right.\n",
    "$$\n",
    "之后我们再将每一组数据中的男女汇总一下，得到这样一组数据\n",
    "$$\n",
    "x_{1} = \\left\\{\\begin{array}{ll}188 \\times 0.999 = 187.812, \\text {男} \\\\188 \\times 0.001 = 0.188, \\text {女}\\end{array}\\right. ,x_{2} = \\left\\{\\begin{array}{ll}158 \\times 0.098 = 15.484, \\text {男} \\\\158 \\times 0.901 = 142.358, \\text {女}\\end{array}\\right. \\\\x_{3}  = \\left\\{\\begin{array}{ll}165 \\times 0.490 = 80.85, \\text {男} \\\\165 \\times 0.510 = 84.51, \\text {女}\\end{array}\\right. ,x_{4} = \\left\\{\\begin{array}{ll}170 \\times 0.52 = 88.4, \\text {男} \\\\170 \\times 0.47 = 79.9, \\text {女}\\end{array}\\right.\n",
    "$$\n",
    "这样的到的四组数据中男和女就更符合现实的情况，因为我们已经把真实的数据掺进去做了一遍计算，那么现在用这四组数据中的男和女来调整一下最开始估计的那六个参数$\\mu_{男}, \\sigma_{男} ^ 2,\\mu_{女}, \\sigma_{女} ^ 2,\\pi_{男},\\pi_{女}$。\n",
    "\n",
    "其中$\\pi_{男},\\pi_{女}$的调整方式就是将四组数据中男，女的可能性相加处以4得到的：\n",
    "$$\n",
    "\\pi_{男} = \\frac{0.999 + 0.098 + 0.490 + 0.529}{4} = 0.529 \\\\\\pi_{女} = \\frac{0.001 + 0.901 + 0.510 + 0.471}{4} = 0.470\n",
    "$$\n",
    "我们再来调整一下$\\mu_{男}, \\sigma_{男}$，这两个参数怎么调整呢，这里直接告诉你一个结论，先不用管他是怎么来的，我们会用就ok了。如果有一组数据$x_1, x_2, x_3, x_4$符合高斯$\\mathrm{N}\\left(\\mu, \\sigma\\right)$分布，那么他们的参数的估计值为：\n",
    "$$\n",
    "\\begin{aligned}&\\mu=\\frac{1}{n} \\sum_{i} x_{i}\\\\&\\sigma^{2}=\\frac{1}{n} \\sum_{i}\\left(x_{i}-\\mu\\right)^{2}\\end{aligned}\n",
    "$$\n",
    "根据这个结论我们拿得到的四组数据更新一下对应的$\\mu_{男}, \\sigma_{男} ^ 2$。\n",
    "$$\n",
    "\\mu_{男} = \\frac{181.812+15.484+80.85+88.4}{0.999+0.098+0.490+0.529}=173.225 \\\\\\sigma_{男} ^ 2 = \\frac{(187.812-173.225)^{2}+(15.484-173.225)^{2}+(80.85-173.225)^{2}+(88.4-173.225)^{2}}{0.999+0.098+0.490+0.529}=19292.733\n",
    "$$\n",
    "根据相同的原则，我们再来更新一下对应的$\\mu_{女}, \\sigma_{女} ^ 2$。\n",
    "$$\n",
    "\\mu_{女} = \\frac{0.188+142.358+84.51+79.9}{0.001+0.901+0.510+0.471}=163.014 \\\\\\sigma_{女} ^ 2 = \\frac{(0.188-163.014)^{2}+(142.358-163.014)^{2}+(84.51-163.014)^{2}+(79.9-163.014)^{2}}{0,001+0,901+0.510+0.471}=21247.897\n",
    "$$\n",
    "然后我们根据更新后的参数再次从头计算一下，因为每次计算都相当于将数据的真实情况掺杂的参数中，最终就能收敛到符合数据分布的真实情况。\n",
    "其实这个例子还有另外一个名称就是高斯混合模型GMM。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [例子2](#content)\n",
    "\n",
    "这里贴一下July$^{[5]}$的硬币案例：\n",
    "\n",
    "两枚硬币A和B，假定随机抛掷后正面朝上概率分别为$P_A$，$P_B$。为了估计这两个硬币朝上的概率，咱们轮流抛硬币A和B，每一轮都连续抛5次，总共5轮：\n",
    "\n",
    "| 硬币 | 结果       | 统计    |\n",
    "| :--- | ---------- | ------- |\n",
    "| A    | 正正反正反 | 3正-2反 |\n",
    "| B    | 反反正正反 | 2正-3反 |\n",
    "| A    | 正反反反反 | 1正-4反 |\n",
    "| B    | 正反反正正 | 3正-2反 |\n",
    "| A    | 反正正反反 | 2正-3反 |\n",
    "\n",
    "硬币A被抛了15次，在第一轮、第三轮、第五轮分别出现了3次正、1次正、2次正，所以很容易估计出$P_A$，类似的，$P_B$也很容易计算出来，如下：\n",
    "\n",
    "$$P_A = （3+1+2）/ 15 = 0.4$$\n",
    "$$P_B= （2+3）/10 = 0.5$$\n",
    "\n",
    "问题来了，如果我们不知道抛的硬币是A还是B呢（即硬币种类是隐变量），然后再轮流抛五轮，得到如下结果：\n",
    "\n",
    "| 硬币    | 结果       | 统计    |\n",
    "| ------- | ---------- | ------- |\n",
    "| Unknown | 正正反正反 | 3正-2反 |\n",
    "| Unknown | 反反正正反 | 2正-3反 |\n",
    "| Unknown | 正反反反反 | 1正-4反 |\n",
    "| Unknown | 正反反正正 | 3正-2反 |\n",
    "| Unknown | 反正正反反 | 2正-3反 |\n",
    "\n",
    "OK，问题变得有意思了。现在我们的目标没变，还是估计$P_A$和$P_B$，需要怎么做呢？\n",
    "\n",
    "> 显然，此时我们多了一个硬币种类的隐变量，设为$z$，可以把它认为是一个5维的向量$（z_1,z_2,z_3,z_4,z_5)$，代表每次投掷时所使用的硬币，比如$z_1$，就代表第一轮投掷时使用的硬币是A还是B。\n",
    ">\n",
    "> - 但是，这个变量$z$不知道，就无法去估计$P_A$和$P_B$，所以，我们必须先估计出$z$，然后才能进一步估计$P_A$和$P_B$。\n",
    "> - 可要估计$z$，我们又得知道$P_A$和$P_B$，这样我们才能估计z，这不是鸡生蛋和蛋生鸡的问题吗，如何破？\n",
    "\n",
    "> 答案就是先随机初始化一个$P_A$和$P_B$，用它来估计$z$，然后基于$z$，还是按照去估计新的$P_A$和$P_B$，如果新的$P_A$和$P_B$和我们初始化的$P_A$和$P_B$一样，请问这说明了什么？\n",
    ">\n",
    "> 这说明我们初始化的$P_A$和$P_B$是一个相当靠谱的估计！\n",
    ">\n",
    "> 就是说，我们初始化的$P_A$和$P_B$，可以估计出$z$，然后基于$z$，反过来估计出$P_1$和$P_2$，当与我们初始化的$P_A$和$P_B$一样时，说明是$P_1$和$P_2$很有可能就是真实的值。\n",
    ">\n",
    "> 如果新估计出来的$P_A$和$P_B$和我们初始化的值差别很大，怎么办呢？就是继续用新的$P_1$和$P_2$迭代，直至收敛。\n",
    "\n",
    "我们不妨这样，先随便给$P_A$和$P_B$赋一个值，比如：\n",
    "硬币A正面朝上的概率$P_A = 0.2$\n",
    "硬币B正面朝上的概率$P_B = 0.7$\n",
    "\n",
    "然后，我们看看第一轮抛掷最可能是哪个硬币。\n",
    "如果是硬币A，得出3正2反的概率为 $0.2*0.2*0.2*0.8*0.8 = 0.00512$\n",
    "如果是硬币B，得出3正2反的概率为$0.7*0.7*0.7*0.3*0.3=0.03087$\n",
    "然后依次求出其他4轮中的相应概率。做成表格如下（标粗表示其概率更大）：\n",
    "\n",
    "| 轮数 | 若是硬币A                                   | 若是硬币B            |\n",
    "| ---- | ------------------------------------------- | -------------------- |\n",
    "| 1    | 0.00512，即0.2 0.2 0.2 0.8 0.8，3正-2反     | **0.03087**，3正-2反 |\n",
    "| 2    | **0.02048**，即0.2 0.2 0.8 0.8 0.8，2正-3反 | 0.01323，2正-3反     |\n",
    "| 3    | **0.08192**，即0.2 0.8 0.8 0.8 0.8，1正-4反 | 0.00567，1正-4反     |\n",
    "| 4    | 0.00512，即0.2 0.2 0.2 0.8 0.8，3正-2反     | **0.03087**，3正-2反 |\n",
    "| 5    | **0.02048**，即0.2 0.2 0.8 0.8 0.8，2正-3反 | 0.01323，2正-3反     |\n",
    "\n",
    "按照最大似然法则：\n",
    "\n",
    "- 第1轮中最有可能的是硬币B\n",
    "- 第2轮中最有可能的是硬币A\n",
    "- 第3轮中最有可能的是硬币A\n",
    "- 第4轮中最有可能的是硬币B\n",
    "- 第5轮中最有可能的是硬币A\n",
    "\n",
    "我们就把概率更大，即更可能是A的，即第2轮、第3轮、第5轮出现正的次数2、1、2相加，除以A被抛的总次数15（A抛了三轮，每轮5次），作为z的估计值，B的计算方法类似。然后我们便可以按照估计新的$P_A$和$P_B$。\n",
    "\n",
    "$P_A = （2+1+2）/15 = 0.33$\n",
    "$P_B =（3+3）/10 = 0.6$\n",
    "\n",
    "设想我们是全知的神，知道每轮抛掷时的硬币就是如本文本节开头标示的那样，那么，$P_A$和$P_B$的估计就是0.4和0.5（下文中将这两个值称为$P_A$和$P_B$的真实值）。那么对比下我们初始化的$P_A$和$P_B$和新估计出的$P_A$和$P_B$：\n",
    "\n",
    "| 初始化的$P_A$ | 估计出的$P_A$ | 真实的$P_A$ | 初始化的$P_B$ | 估计出的$P_B$ | 真实的$P_B$ |\n",
    "| ------------- | ------------- | ----------- | ------------- | ------------- | ----------- |\n",
    "| 0.2           | 0.33          | 0.4         | 0.7           | 0.6           | 0.5         |\n",
    "\n",
    "看到没？我们估计的$P_A$和$P_B$相比于它们的初始值，更接近它们的真实值了！就这样，不断迭代不断接近真实值，这就是EM算法的奇妙之处。\n",
    "\n",
    "可以期待，我们继续按照上面的思路，用估计出的$P_A$和$P_B$再来估计$z$，再用$z$来估计新的$P_A$和$P_B$，反复迭代下去，就可以最终得到$P_A = 0.4$，$P_B=0.5$，此时无论怎样迭代，$P_A$和$P_B$的值都会保持0.4和0.5不变，于是乎，我们就找到了$P_A$和$P_B$的最有可能的估计值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [问题思考](#content)\n",
    "\n",
    "看完这两个例子不知道大家有没有更清楚EM算法的整个流程，但是你可能会有以下疑问🤔️：\n",
    "\n",
    "- 身高的例子中最开始的分布情况真的是瞎猜的么？\n",
    "- 硬币的例子中通过z估计$P_A$和$P_B$是采用什么方式估计的呢？\n",
    "- 硬币中的隐变量z代表什么？在身高的案例中哪个是隐变量呢？\n",
    "\n",
    "这几点疑问可以这样来解释：\n",
    "\n",
    "- 其实在身高的那个案例中我想大家也能发现，我们猜测的那几个参数$\\mu_{男}, \\sigma_{男} ^ {2},\\mu_{女}, \\sigma_{女} ^ {2}$还是很符合真实情况的，就像身高188，我们计算之后确实是男性的可能性会更大。这四个参数也不是我胡乱猜测的，而是我在网上查到的人类身高分布情况，然后根据国家统计局的数据做的一个预设。这一步数学化来说就是先验概率。\n",
    "\n",
    "- 在抛硬币的案例中，我们是已经观察到了几正几反，然后反推是哪一个硬币出现这种情况的可能性大。这个用数学化来说就是极大似然估计。\n",
    "\n",
    "- 硬币中的隐变量z就是我们不知道这个硬币到底是A还是B，要是能知道这个硬币是A还是B那么我们就可以直接算出来A和B正面朝上的概率了。在身高例子中，我们不知道当前这个志愿者到底是男生还是女生，如果知道的话，那么我们的男女比例$\\pi_{男},\\pi_{女}$就已知了。而且要是知道哪几个是男性的话，我们直接用男性身高数据的参数估计就直接能得到$\\mu_{男}, \\sigma_{男} ^ {2}$，也就不用再那么复杂的计算了。\n",
    "\n",
    "所以EM算法的流程就是这样：\n",
    "\n",
    "1. 先给出数据的先验概率，来假设参数，这个参数我们统一用$\\theta$来表示\n",
    "2. 然后根据当前的数据和参数$\\theta$，来计算一下这个隐变量的可能的值(这个值用数学化表述就是隐变量在条件是的情况下的数学期望)。\n",
    "3. 根据计算出来的隐变量可能的值，反过来用极大似然估计的方法推倒发生这样的情况下，参数$\\theta$的最优解是多少？\n",
    "4. 通过一次计算我们肯定不能知道$\\theta$是不是最优值，我们还要继续执行2，3两步，直到参数已经变化不大了，我们就可以判定这个$\\theta$收敛到了一个近似的最优解。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [数学推导](#content)\n",
    "\n",
    "假定我们现在有训练集$\\left\\{x^{(1)}, x^{(2)}, \\cdots, x^{(m)}\\right\\}$，包含m个独立的样本，我们现在希望从中找到该组数据的模型的参数$\\theta$。为了求解这个问题，我们先取参数的对数极大似然\n",
    "$$\n",
    "l(\\theta)=\\sum_{i=1}^{m} \\log p(x^{i} ; \\theta)\n",
    "$$\n",
    "但是我们不要忘了，我们还有隐变量$z=$$\n",
    "\\left(z^{1}, z^{2}, \\ldots z^{m}\\right)\n",
    "$，我们也要把z添加到似然函数当中去\n",
    "$$\n",
    "l(\\theta)=\\sum_{i=1}^{m} \\log \\sum_{z} p(x^{i}, z ; \\theta)\n",
    "$$\n",
    "正常极大似然估计是最大化上面那个式子，求导令其得0求得参数值，但是这里明显无法直接求出参数的。所以我们需要一些技巧来求解这个似然函数。\n",
    "\n",
    "我们令$Q_i$代表隐变量$z$的一个分布，现在我们来改造一下似然函数(这里就是上下同时乘以$Q_i$)\n",
    "$$\n",
    "\\begin{aligned}l(\\theta)=\\sum_{i=1}^{m} \\log \\sum_{z} P\\left(x^{i}, z ; \\theta\\right) &=\\sum_{i=1}^{m} \\log \\sum_{Z^{(i)}} Q_{i}\\left(z^{(i)}\\right) \\frac{P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{Q_{i}\\left(z^{(i)}\\right)}           \\\\& \\geq \\sum_{i=1}^{m} \\sum_{z^{(i)}} Q_{i}\\left(z^{(i)}\\right) \\log \\frac{P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{Q_{i}\\left(z^{(i)}\\right)}\\end{aligned}\n",
    "$$\n",
    "这里我们还要注意一下这个式子\n",
    "$$\n",
    "\\sum_{z^{(i)}} Q_{i}\\left(z^{(i)}\\right)\\frac{p\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{Q_{i}\\left(z^{(i)}\\right)}\n",
    "$$\n",
    "这个代表什么意思呀？这个不就是$\\frac{p\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{Q_{i}\\left(z^{(i)}\\right)}$的期望么？（不理解的可以翻看一下期望的公式）\n",
    "\n",
    "这里用到了<a href = \"#Jensen不等式\">Jensen不等式</a>，将等号变为大于等于号，而且因为这里的对数函数为凹函数，所以得出这个结论\n",
    "$$\n",
    "f(E(x)) \\geq E(f(x)) \\quad \\text { 如果 } f(x) \\text { 是凹函数 }\n",
    "$$\n",
    "但是我们还要考虑的一点是，什么情况下才能取到这个等号。\n",
    "$$\n",
    "\\frac{P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{Q_{i}\\left(z^{(i)}\\right)}=c, c \\text { 为常数 }\n",
    "$$\n",
    "我们只有令上面这个式子等于c也就是为一个定值时才能取到等号。而且$Q_i$是一个分布，所以满足\n",
    "$$\n",
    "\\sum_{z} Q_{i}\\left(z^{(i)}\\right)=1\n",
    "$$\n",
    "这样结合上面的公式，我们得到这样一个结果\n",
    "$$\n",
    "\\left.Q_{i}\\left(z^{(i)}\\right)=\\frac{P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{\\sum_{z} P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}=\\frac{P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{P\\left(x^{(i)} ; \\theta\\right)}=P\\left(z^{(i)} | x^{(i)} ; \\theta\\right)\\right.\n",
    "$$\n",
    "也就是说令$Q_i$为$\\theta$的一个条件概率时就可以满足上面的条件。到这里就是EM算法中的E。\n",
    "\n",
    "现在如果我们极大化这个式子\n",
    "$$\n",
    "\\theta := \\arg \\max _{\\theta} \\sum_{i=1}^{m} \\sum_{z^{(i)}} Q_{i}\\left(z^{(i)}\\right) \\log \\frac{P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)}{Q_{i}\\left(z^{(i)}\\right)}\n",
    "$$\n",
    "也就意味着在寻找似然函数$l(\\theta)$的下界。发现没有我们将原来复杂的一个问题简单化了，现在我们尽可能的让上面的式子求其最大值，直到这个最大值等于原本的似然函数$l(\\theta)$，我们也就间接的得到了似然函数$l(\\theta)$的极大值，有点绕，但是我们能得到一个信息，就是可以求似然函数的极大值了。这个下界我们不妨用$J$来表示，且$J$与$Q_i$有关。\n",
    "\n",
    "![theta](../img/theta.jpeg)\n",
    "\n",
    "固定$\\theta_{1}$，调整$Q_i$使下界$J$与似然函数在$\\theta_{1}$点处相等，然后固定$Q_i$，调整$\\theta$使得下界$J$达到最大值，此时得到新的$\\theta_{2}$，然后再固定$\\theta_{2}$，调整$Q_i$使得下界$J$与似然函数相等，重复这个过程，直到收敛至似然函数的最大值。\n",
    "\n",
    "现在我们把上面式子中的常数项去掉再看一下\n",
    "$$\n",
    "\\theta := \\arg \\max _{\\theta} \\sum_{i=1}^{m} \\sum_{z^{(i)}} Q_{i}\\left(z^{(i)}\\right) \\log P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)\n",
    "$$\n",
    "上面这个式子就是EM算法中的M。\n",
    "\n",
    "至此整个EM算法的数学公式推导过程已经完毕了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [EM算法流程](#content)\n",
    "\n",
    "参考自EM算法原理总结$^{[4]}$。\n",
    "\n",
    "EM算法的整体流程如下：\n",
    "\n",
    "输入：数据$\\left\\{x^{(1)}, x^{(2)}, \\cdots, x^{(m)}\\right\\}$，最大迭代次数J\n",
    "\n",
    "（1）先验给出模型的初始化参数$\\theta$\n",
    "\n",
    "（2）for i from 1 to J:\n",
    "\n",
    "​    （a）E: 计算期望\n",
    "$$\n",
    "\\begin{aligned}&\\left.Q_{i}\\left(z^{(i)}\\right)=P\\left(z^{(i)} | x^{(i)}, \\theta^{j}\\right)\\right)\\\\&L\\left(\\theta, \\theta^{j}\\right)=\\sum_{i=1}^{m} \\sum_{z^{(i)}} Q_{i}\\left(z^{(i)}\\right) \\log P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)\\end{aligned}\n",
    "$$\n",
    "​    （b）M: 最大化$L\\left(\\theta, \\theta^{j}\\right)$，得到$\\theta^{j+1}$：\n",
    "$$\n",
    "\\theta^{j+1}=\\arg \\max _{\\theta} L\\left(\\theta, \\theta^{j}\\right)\n",
    "$$\n",
    "​    （c）如果已收敛，则算法结束，否则继续回到步骤（a）进行迭代\n",
    "\n",
    "输出：模型参数$\\theta$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [代码实现](#content)\n",
    "### [二维高斯分布](#content)\n",
    "\n",
    "多维变量$X=(x_1,x_2,...,x_n)$的联合概率密度函数为：\n",
    "\n",
    "$$\n",
    "f(X)=\\frac{1}{(2 \\pi)^{d / 2}|\\Sigma|^{1 / 2}} \\exp \\left[-\\frac{1}{2}(X-u)^{T} \\Sigma^{-1}(X-u)\\right], X=\\left(x_{1}, x_{2} \\dots x_{n}\\right)\n",
    "$$\n",
    "\n",
    "其中：\n",
    "\n",
    "　　- $d$：变量维度。对于二维高斯分布，有d=2;\n",
    "  \n",
    "　　- $u=(u_1,u_2,...,u_n)$：各位变量的均值；\n",
    "  \n",
    "　　- $Σ$：协方差矩阵，描述各维变量之间的相关度。对于二维高斯分布，有：\n",
    "\n",
    "$$\n",
    "\\Sigma=\\left(\\begin{array}{ll}\n",
    "\\delta_{11} & \\delta_{12} \\\\\n",
    "\\delta_{21} & \\delta_{22}\n",
    "\\end{array}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [数据解析](#content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.patches import Ellipse\n",
    "from scipy.stats import multivariate_normal\n",
    "plt.style.use('seaborn')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_cluster(X, Mu_true, Var_true, color, ang = 0):\n",
    "    plt.figure(figsize=(10, 8))\n",
    "    plt.axis([-10, 15, -5, 15])\n",
    "    ax = plt.gca()\n",
    "    plt.scatter(X[:, 0], X[:, 1], s=5, c=color)\n",
    "    plot_args = {'fc': 'None', 'lw': 2, 'edgecolor': color, 'alpha': 0.5}\n",
    "    ellipse = Ellipse(Mu_true, 3 * Var_true[0], 3 * Var_true[1], angle=ang, **plot_args)\n",
    "    ax.add_patch(ellipse)\n",
    "    plt.show()\n",
    "\n",
    "nums = [400, 600, 1000, 500]\n",
    "true_Mu = [[0.5, 0.5], [5.5, 2.5], [1, 7], [9, 4.5]]\n",
    "true_Var = [[1, 3], [2, 2], [6, 2], [1, 3]]\n",
    "var = [np.diag(true_Var[0]), np.diag(true_Var[1]),np.diag(true_Var[2]),np.array([[1,-1],[-1,3]])]    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一个例子\n",
    "$$\n",
    "u=(0.5 \\quad 0.5), \\Sigma=\\left(\\begin{array}{ll}\n",
    "1 & 0 \\\\\n",
    "0 & 3\n",
    "\\end{array}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAHUCAYAAADvF/aYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de3Rc5X3v/8+eiyxfhG0cYWMCxgh4CnG4GTAQAhgnJSGcKNinqZyT5DgJIe2v/ZUm5/xa2nAOJ2vRNslK0+Omt0VJj3sJFhSbKKS5UAwhMQTMaXCIUvIYZCg2toVQfMXWZWb274+tLW+N5rLnsjWzpfdrLS+kmT17ntGOnI+f57u/j+O6rgAAABCdRKMHAAAAMN0RuAAAACJG4AIAAIgYgQsAACBiBC4AAICIEbgAAAAilgp7oDFmlaQvWWtvMMZcJukRSS+NPf3X1toHAsfOlvRPkk6TdFTSf7XWDtRv2AAAAPERKnAZY35P0sckvTX20GWSvmqt/dMiL/lNST+z1v4vY0yXpLsk3VHrYAEAAOIo7JJin6S1ge9XSvqAMeaHxpivG2Pa8o6/VtL3xr7+rqT31DZMAACA+Ao1w2Wt3WKMOTvw0A5J91lr/80Y83lJd0v674HnT5F0eOzro5Lml3sP13Vdx3FCDRoAAKDBKgotoWu48jxsrT3kfy3pa3nPH5Hkz3q1STqkMhzH0cDA0SqHg0Zrb2/j+sUU1y7euH7xxbWLt/b2/MW90qq9S/H7xpgrx75eI+nf8p5/StLNY1+/X9KPqnwfAACA2Kt2hus3Jf2FMWZE0gFJt0uSMeZRSbdI+mtJf2+M2S5pRNJH6jBWAACAWHJc1230GHwuU6vxxdR4fHHt4o3rF19cu3hrb2+rqIaLxqcAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABCxVNgDjTGrJH3JWnuDMeYSSV+TlJU0LOnj1tr+vOOfl3R47NtXrLWfqNOYAQAAYiVU4DLG/J6kj0l6a+yhjZL+X2vtTmPMZyT9vqTPBY5vlSRr7Q11HS0AAEAMhZ3h6pO0VtI/jn3fZa3dHzjHUN7xF0uaY4x5dOz5P7TWPlPuTdrb20IOB82I6xdfXLt44/rFF9du5ggVuKy1W4wxZwe+3y9JxphrJP22pOvyXnJc0lck3SfpPEnfNcYYa22m1PsMDBwNP3I0lfb2Nq5fTHHt4o3rF19cu3irNCyHruHKZ4z5dUmfl/QBa+1A3tO7JL1srXUl7TLGDEo6XdKeat8PAAAgrqq6S9EY81F5M1s3WGt3Fzjkk5L+dOzYpZJOkbS/wHEAAADTXsUzXMaYpKQ/l/SapK3GGEl60lp7tzHmHyTdJenrkjYZY7ZLciV9stxyIgAAwHQVOnBZa1+VdNXYt6cWOebjgW8/Uv2wAAAApg8anwIAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAELFU2AONMaskfclae4Mx5lxJmyS5knol/Za1Nhc4drakf5J0mqSjkv6rtXagngMHAACIi1AzXMaY35N0n6TWsYe+Kukua+27JTmSOvNe8puSfjb2/D9Iuqs+wwUAAIifsEuKfZLWBr5fKenJsa+/K+k9ecdfK+l7JZ4HAACYMUItKVprtxhjzg485Fhr3bGvj0qan/eSUyQdLvF8Qe3tbWEOQ5Pi+sUX1y7euH7xxbWbOULXcOXJBb5uk3Qo7/kjY48Xe76ggYGjVQ4Hjdbe3sb1iymuXbxx/eKLaxdvlYblau9SfN4Yc8PY1++X9KO855+SdHOJ5wEAAGaMame4/pukvzXGtEh6UdJDkmSMeVTSLZL+WtLfG2O2SxqR9JE6jBUAACCWHNd1yx81NVymVuOLqfH44trFG9cvvrh28dbe3uZUcjyNTwEAACJG4AIAAIgYgQsAACBiBC4AAICIEbgAAAAiRuACAACIGIELAAAgYgQuAACAiBG4AAAAIkbgAgAAiBiBCwAAIGIELgAAgIgRuAAAACJG4AIAAIgYgQsAACBiBC4AAICIEbgAAAAiRuACAACIGIELAAAgYgQuAACAiBG4AAAAIkbgAgAAiBiBCwAAIGIELgAAgIgRuAAAACJG4AIAAIgYgQsAACBiBC4AAICIEbgAAAAiRuACAACIGIELAAAgYgQuAACAiBG4AAAAIkbgAgAAiBiBCwAAIGIELgAAgIgRuAAAACJG4AIAAIgYgQsAACBiBC4AAICIEbgAAAAiRuACAACIGIELAAAgYgQuAACAiBG4AAAAIkbgAgAAiFiq2hcaYzZI2jD2baukSyQtsdYeGnv+zyW9S9LRsWM6rbWHqx4pAABATFUduKy1myRtkiRjzF9K+js/bI25TNJN1to3axkgAABA3NW8pGiMuVzSO6y19wYeS0g6T9K9xpinjDGfrPV9AAAA4qrqGa6AP5T0hbzH5kr6mqSvSkpKesIY83+ttS+UOlF7e1sdhoNG4frFF9cu3rh+8cW1mzlqClzGmAWSfsVa+0TeU8clbbTWHh877nFJF0sqGbgGBo6WehpNrL29jesXU1y7eOP6xRfXLt4qDcu1LileJ+mxAo+fL2m7MSZpjElLulbST2p8LwAAgFiqdUnRSNo9/o0xn5P0srX2W8aYb0h6RtKopH+w1v68xvcCAACIJcd13UaPwecytRpfTI3HF9cu3rh+8cW1i7f29jankuNpfAoAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAQAARIzABQAAEDECFwAAQMQIXAAAABEjcAEAAESMwAUAABAxAhcAAEDECFwAAAARI3ABAABEjMAFAAAQMQIXAABAxAhcAAAAESNwAYhMT09S69e3qqcn2eihAEBDpRo9AADTV3d3Wtu2pSVJnZ3ZBo8GABqHwAUgMl1doxP+CwAzFYELQGQ6O7PMbAGAagxcxpjnJR0e+/YVa+0nAs99WtJnJGUk3WOt/XYt7wUAABBXVQcuY0yrJFlrbyjw3BJJvyPpckmtkrYbY/7VWjtc7fsBmHqZjLRnj6PXXkvoxAkpm3XU0uJqwQJX55zj6tRTXTlOo0cJAM2vlhmuiyXNMcY8OnaeP7TWPjP23JWSnhoLWMPGmJclXSTpuVInbG9vq2E4aDSuX3zlX7vdu6UdO7z/jowUfs2OHdLChZIx0rXXSvPmTcFAURC/e/HFtZs5aglcxyV9RdJ9ks6T9F1jjLHWZiSdopNLjZJ0VNL8ciccGDhaw3DQSO3tbVy/mApeu0OHpB/8IKVdu052jDntNFfLl+e0YIGrREIaHZX2709o9+6E9u6V9u6Vtm+Xrr46o5Urc0rSAWJK8bsXX1y7eKs0LNcSuHZJetla60raZYwZlHS6pD2SjkgKjqRN0qEa3gtAhFxXevbZpJ5+OqlMRmppkVatymrFiqzaCv6dklMuJx044OiZZ5J6+eWEfvCDlF54wdUtt2S0ZIk71R8BAJpaLYHrk5LeKen/McYslTertX/suR2S/miszmuWpAsk9dYyUADRyOWk738/qRde8KamLrwwp+uvzxQJWiclEtLSpa7Wrs1o925HTzyR0uCgo+7utD74wVGdcw6hCwB8tXSa/7qkBcaY7ZIekBfAfscY80Fr7QFJfy7pR5Iel/R5a+1QzaMFUFcjI1J3t/TCC0ml09Ktt2Z0yy3lw1a+c85xtWHDqN7xjpxGRqStW9P62c/YyAIAfI7rNs2/Ql3WsuOLWoT4GR2VHnwwrUOHWpXLDWvt2lGdcUZtfx+4rvSjHyX1zDPebNmaNV5dF6LD7158ce3irb29raJ7tPknKDADua70ve+l9PrrjubPlz7ykdrDliQ5jnTddVm9970ZSdLjj6e0ezd9IwCAwAXMQM8+m9SLLybU0iL9l/8iLVpU35nuSy/N6V3vysp1pUceSWtwkNAFYGYjcAEzzEsvJfTDHyblONItt4zqtNOieZ9rrsnKmJyGh6WtW1M6cSKa9wGAOCBwATPIkSPSd77j3Zz87ndnde650dVwOo70/vdndNpprg4edPT977N1K4CZi8AFzBCuK/3rv6Y0PCydd15Oq1ZFv6l0S4t0662jammRdu1KTGioCgAzCX/7ATPEL36RUF9fQq2t0nvfm5myPRDnz5euu84ron/ssaSGamwQ09OT1Pr1rerpoaU9gPggcAEzwPHj0rZtXkC5/vrMlO97eOmlOZ1xhqtjxxw9+WRtQam7O61t29Lq7k7XaXQAED0CFzADPPlkSsePOzrrrJwuumjq+2I5jnTTTRklk9JPf5rUnj3VT691dY1qzZpRdXWN1nGEABAtAhcwzQ0MOOrtTSiZlH71V6duKTHf297mjteNPflkStX2XO7szGrz5iF1dkZfgwYA9ULgAqa5p55KynWliy/O6tRTGzuWK6/Mas4cV/v2OTREBTCjELiAaay/39Ejj6S0dWtKg4O1n6/WgvWWFo3Pcm3fXv0sFwDEDYELmGaCoWj79qR6exPavTupb36z9iLzehSsX3JJTvPmuervd/TSS/wVBGBmoBMhMM34oWhoSLr66pwuvjirZctydSky989Ry7nSaemqq7J67LGUtm9P6txzc0qQuwBMcwQuYJrxw9Dy5d7diB/9aEbXXVefAvPOzmxditUvuiinHTtcvfmmo76+hM47b+rvnASAqcS/K4FpprMzq7/5myEtXCglk9JllzXf3XyplLRypTeu55/nryEA0x9/0wHT0M6d3p2J55+fm/Imp2GtWJFTOi29+mpCv/xlo0cDANEicAHTTCYjvfCCdxdhs81uBQv6Z8+WLrjAG9/OnWzTA2B6I3AB08wvfpHQiRPSaae5Wrq0ufou5N/leOmlXu1Wb29SIyONHBkARIvABTSpante+bNFl12WbVhX+WLyt+VZvNjVGWe4GhqSXnyRv44ATF/8DQc0qWp6Xr3xhqN9+xy1tkoXXFDbnX+lAl+1YbDQtjyXXOJ9/dOfsqwIYPqiLQTQpKrpefXv/+79G+pXfiWrdI19Tv3AJ2lSK4hSz1XKmJwee0w6cMDR4KCjRYuaaxkUAOqBGS6gSVW6SfM3v5nU5z8/S9YmdOGFtfe1yl/+C/tcpVIp725KKdyyYq3bCwFAIzDDBUwTmzal9dJLXgg544zaZ4lKNTmtVwNU3wUXZPWznyX04osJvetdpWvP6jm7BgBThRkuYJq4/PKsli/P6pZbRgsGlmaaGcofy1lnuZo719XBg47eeKN0pX89Z9cAYKowwwVMA7mctHChtG5dRh//eKbgMc00M5Q/lkceSeqb30zprLNc/eIXCS1eXHx89Z5dA4CpQOACpoHXX3f01luOFixwtXhx4eXEemw8XUhPT1Ld3Wl1dY2GDkL5Y+nuTuunP03pyJGsdu1K6Lrrmq+lBQDUgsAFTAMvveRVB5x/fq5oUIliZqinJ6k775ylwcHk+HuEkT+Wrq5Rua506qnesuLAgKPTTuNuRQDTBzVcQMSmonbqlVe8X+WOjtrvTqxEd3dag4NJLVqUrXrmzJ8hW79+VLfc4oUw//MAwHTBDBcQsahrpw4flgYHHc2apSnfyie4NFjtZwv+fP7n/xxRb29Cr77qaNWqug0TABqOwAVELKraKd+rr3qzQcuW5ZSc4hsQ67FMGfz5LFvmLYnu3ZvQyIjU0lKPUQJA4zFvD0SsUAPTei4z+stvy5eHW06Mcomz0LnLvV/w5zNnjnT66a6yWWnPHqrmAUwfBC6gAarZJ7GQbFb6j//wfo3PPjtc4Cr03vUKYYXOXeln9T8HdVwAphOWFIEGqNcy4759joaHpUWLXM2fX/1716vOrNC5K/2sy5fn9PTTybHARb8tANOD47pNc+u1OzBwtNFjQJXa29vE9Zt6P/xhUs88k9Tll2d1443VhZP29jbdd99xdXen1dGRU19fYkJ/rFoK4quRy0l/8RctGhqSPv3pES1cOGVvHUv87sUX1y7e2tvbKqp7YIYLiLHXX/d+35ctq60dhF/8vn596/hMl6Qp60yf3zz1rLNy2rUrob17E1q4cGpbXQBAFAhcQEzlctKBA16d0+mn1zZT7Qcer4/XaMElwfxji818Bc/lz5aVC2z5S5qnn+5q1y5p/35H73xnTR8NAJoCgQuIqYEBR6Oj0sKFrubMqe1cJwPPqDZvHhp/vFBQKlfv5T+/c2c2dAf6/DqvpUtzkpLat486LgDTA4ELiKl9+7zlxHo0O62ksL3csf7j3gxXLtQ58/t5LVniKpHwQiX9uABMBwQuIKZef91bTvRmg2rjBx6/PUSpZcByzU7r0Qw1nZZOO83VgQOO9u93tGxZ09zcAwBVodENEFP799dvhstXr/5g9eAHyf37+WsKQPzxNxkQQ8ePSwcPOkqnpfb2+gWurq5RrVkzGtk2RIUUa7rq3wjgL50CQJyxpAjEkD+7tWRJTok6/rOpHsuBlSpWhO/PcO3b58h1JYfcBSDGmOECYsi7e6++y4lBUe63mK/YrNqCBdKcOa6OH3d0+HDkwwCASBG4gBpMZTAJqvQOxUrHWa6Wq56fu9Dm3pI3o+V/Pj9gAkBcsaQI1KBeexBWIpc7WUh++umVbVgd/N6fUdq6VVq7Njlh/OVaP2zc2KLe3pT6+x11dp6o6nOEsXSpq5df9gLmhRdG9jYAEDkCF1CDem1CXYk33/R6U82f72revHCvCY4zP3xt2yaNjKQnBK5G1HIV4gVKGqACiL+qApcxJi3p7ySdLWmWpHustd8KPP85SZ+SNDD20Gestba2oQLNpxHB5M03veXExYvD128VGqcfwlpa0lq7trLAeMcdI+rudosGzXLb/4Tlf8bBQQrnAcRbtTNcH5U0aK39mDFmkaTnJX0r8Pxlkj5urf23WgcIYKJDh7zUsXBhdQXz+eHrttuk++5T2Yanpc6RH7DqtdTa2nqycP7YMamtrepTAUBDVVuJ+s+S/kfg+0ze8ysl/YExZrsx5g+qfA9gWqh3Yf3Bg7UFrkJqbXjqv37jxhatX9+qjo6c1qwZVUdHrubPvnCh91//cwNAHFU1w2WtPSZJxpg2SQ9JuivvkG5JfynpiKSHjTG3WGu/Xe687e388zXOuH6Fbd3q1Um1tKR12221ny+blebOlTo6Zqm9vfzxDz4obdokbdggffjDhY+5/fa0WlqkDRvSam+vPHTdfru33+H+/amxzyo99ph08821f/Zly6RDhyTHCfd5ZyJ+9+KLazdzVF00b4w5U9LDkv7KWnt/4HFH0v+21h4e+/5fJF0qqWzgGhg4Wu1w0GDt7W1cvyLWrk1qZMSrkxoYqL3e67XX0jp+3FEuN6KBgfLH33tvq7ZtS2tkZFSrVw9Ner69vU2rVx/V6tXe92HOmW/1au/PXXe1aM+elM48M6OBgZG6fPZEIqm33kpq9+6szjyzunPUq6asGfG7F19cu3irNCxXWzS/WNKjkn7bWrst7+lTJPUaYy6Q9JakG+UV2AMzUj0L64eGpOPHvS19qrlDMWp9fQkNDibV1+e1q6jHZ1+wwFs69WvXqtGI9h0AEFTtDNcfSloo6X8YY/xarr+VNNdae68x5g8lPSFpWNI2a+13ah8qAD90zJ/vhr5jLxh6Kp3p6elJauPGFknenYnlzhNFuPNr1Wqp4WpE+w4ACKq2husOSXeUeP4fJf1jtYMCUJgfOk49tbqC+Upnerq70+rtTY197Y6/pth5ys1oFQpq5UJgcIar2tYQzdJXDMDMReNTIEb8GS4/hFSq0pmerq5R9fc7k15T7YyRH9R27kxIGp7QQiL4WNDs2d6fEyekt94Kv5QKAM2EDcqAGKmmJUSwLUX+voU9PUndfLOKtm3o7Mzq8cdP6PHHT0yaySq0/2Gp95a8gLZoUVaDg0l99rOtuvHG2eroyI0/VqwtRT2WFQGgkQhcQIwcOuT9t5IZrlI9trq70/rud1V1/61K37uzM6svfnFYixZldexYQr29KfX1JfTFLw5rzZrRCTNmwbBWSeF8ozYUB4BSWFIEYiR/hqtYUXtQqeW/rq7RCVv71Lt9QqH39s47PD7u4HsFg1mwTuy3f9t7fZgZLu5IBNCMCFxATAwPS2+95SiVkh5/PKkHHkirv98pWNQeVKpgvLMzq9tu03iPrDBhpZJQVui9/dfnB8T89w6GtUpmuLgjEUAzInABMRFsCfHAA144WbEioxUrvJ216hEwwoSVsDNIxYJZsdfnv3cwrO3b5332MDNc3JEIoBkRuICYCLaECIaTSsJFudmpMGGlUCgrdN6wwarUe/vn9Zc8a2kNAQCNROACYiLYEmL16vLBKEwI6ulJautWb/sh//lCYazQuTZubNEXviAtWOAd4y9t+s9XEqyKCY736qtzGhqiNQSAeCJwATHQ05PU177WomXLXP3qr2ZCvabQDFN+CPKOkUZGvOOKLRVu3Nii3t6U+vsddXaeGP9ekvbulVasyEy6y9APVv5dg5V0t/fDXXC8R44ktH+/o0OHHM2bV10fMgBoFAIXEAPd3Wm98EJKR49mNX9+uLBR7A7B/OW94F2K+ccX0tOT1Kuveh1lZs3K6bzzckXvkPTHXsldg8FwF+z/9cgjznjgevvbCVwA4oXABcRAV9eo9uxJyJic5s4N95pSS3fBWaTvfOfkXYrSxNYMvjvuGFF3t1c71t2d1rFjCS1a5PXUClPz1d/vqL/fGW++Wo25c72Qdfx4VS8HgIYicAFVqnfPqlI6O7Pau3dUb73laPbsibM71YwjOOt0220nzxFsM1Fuj8Sw7+f11HLHGqC64+9f7PXBcBfU2ur9d3g4XPPTqbo2ABAGgQuo0lQ22HRdaWjICxp+8AgzjmLBY+JyY3r8HIVqsQqdp9LPG3y/QuMNc/7WVi+sDQ2Vfz+anwJoNgQuoEpT2WAzk5GyWSmV8v6EHUex4FGolsv/b6GAUm6D6WKCQWrz5olJKTjeMOf3g6YfPEvJ/5kw4wWg0QhcQJWmssGmP6sza5Y7qQdVqXGEDR7lPktX16h27kyMbzBdzdKl/5piy5P++TdubBlfUgwe589wnThR/n3z34MZLwCNRuACYuDEicLLib6wQaqa5ceTxw6PP19KsbYOpQTP39/vTOoV1t2dHm+HEaaGKx/b/QBoNAIXEAPDw95/8wvmfWGX/KpZfvSFndELnmfz5qHQM0rBvl3Bonn/fKOj0pVX5vTccwl95zvh+3oBQDMgcAExUKxg3ldqyS9swXu9ZoFqPU+x+rJbbx3Vq68m9dxzSfX1JcePDYMlRQCNRuACYuBkDVfh5/OX/IIhK2zYqFdNWtj+X5XOfOVy0le+ktSFF2a1bFmuokDHkiKARiNwATHgF4oXW1KUJgad9etbx0NWvRqP5isVnoo9V8tMUyLhBU5jXP3O7wwXne0rJPiz4Y5FAI2QaPQAgJnI31+wpycZ6ni/UDxsyOjqGh3vp9XZmdXixa56e1PjXeSLjeeuu1pCj8sPT4XOWei5np6k+vsdrViRqXqmqZJeXMWUGjcARIUZLmAKleroXmrmxQ8YfuAop1SfrUL8EPLkkzllMokJ4yr0GTZubNGhQyoangq9X3d3Wr29Ka1ZU35GrNjPorVVOnzYr2mrbj9FlhcBNAKBC5hCxTq69/QkdeedszQ4WLgY3C+aL1bDVU65+qyOjtx42Fq0KDupDix/WdAPi1Km7B2N/nk6OnKSJnex988ZXGostvQ4a5YryQnVi6uYqeyfBgA+AhcwhYp1dO/uTmtwMDkedvKdOCFZm9BTT7Xok5882S6hXnVIfX2J8bDlb0gdrAPLny17/PGUXNfRwED5qoST4Wl0vIjfu9MwMT7+/FmnYrNQs2d7/61mhovaLQCNROACplCx2ZVyW+sMDzvq7U3olVeS2rUrodNPdwtuMl2tQu9frEt9R0dOLS2uhocdtbfnQp+7oyM3Pou3c2d2wmxe/s+l3H6Kfl+yStAaAkAjEbiAJlBumWtoSFqxIqdDh6TBwaROP/3kkmQ9Zm4KvX+xLvU7d2Y1POzNht1990joc69f3zo+i7duXUZ9fRNbO4T5HJXsp5iP2i0AjUTgAmLgxAlHxuR0002jevjhiaGk2NJfvQVnqvywFLbfVvAOxTvuGCm5QXapz+HXsFVTw0XtFoBGoi0E0ORc9+QS2n/+z9lJ2+UEW0DUolyris5O772vuMJ77xHqJ1UAACAASURBVOeeK358fusFv9B+8WJ3PIDlv7bU5/CP37HDm9nyZ7gqba8BAI3CDBfQ5EZGvNDV0uI1/8xXr5mbsDVOwaXFYndVliuCL/RepT6Hf/yxY9K7350bD6DUZQGICwIX0OR6epLasiWlSy+NNlCErXEqtLSYr1xn90rrqfzjrr8+oyNHEtqxI6EHHmgt2WoCAJoJgQtocg89lNYrrySVrPOqmR+Ebr9dWr06/OsqnVGrdDar1Hu+/LKjrVsT+vGPk/r5z1OSRrV5cw1t5wFgilDDBTS5detGtXx5VpdcUt8ZLj8Ibdo08ft6b3kTpsYsvxarWG2Wv6R61VXZutStAcBUYYYLaHI335zVo4/mtHNnUnfd1TKhYWgt/LCyYUN6wvf1DDFhW1YU6zS/c2dC0vCk1156aU5XXZUdD4fUbwFodgQuoMnlchpvenrwoMYL1aXaus37y3RPPJHW+vWt6uqq7/Jcue2KggoV1+/cmdDgoBfY/Nc6Y+23XJeCeQDxwpIi0GTyl9Nc12t6et55XsNQfymtXkuAmzYp1HnCtGC4664WXXDBHN11V0vR7YoKncdvORGs8friF4e1YkVG/f3O+LH+kqLrVtcOgzYSABqFGS6gyeTP3ORykjHeEtqnPz05XJTrNl9uWW/DBmlkpHxwKTajdNddLdqyJaV16zLasiWlwcGktmyRvvhFr3dDR0duwtJf2Jkp71h3LAx6/buCM1zVtMNgVgxAoxC4gCaTv7zmul7KcPJ2swkGjlLd5jdubFFvb0r9/Y46Oye3aP/wh6XVqycuJVbSysEPWfff72jBgpzmzctp3brMhC19/LFJGu84H2Zbn/z39Ge4cjmnqi2N2N4HQKMQuIAmkz9zkxvbHzo/cAXVI0gEZ6r6+hKhWzl4M1vS7Nmu9u5Nac2aUd1zz8k9FoNj8zvOr1lzcsuf7u60+vudgptx57+n/zN4/nlH//Iv4erDgtjeB0CjELiABqt18+lyr7/jjhF1d7tllx7vvz+tY8cSuv9+R3/2Z96MV5gAd8UVWfX1JcYaobqTXlMo5HR05LR+fet40FqxIhOqHssPXDt2JAvWhwFAsyJwAQ2WX1eUH4pKzWzlv97/PrhpdPD7UkuPCxbkdOxYQgsW5CqaCTr5/uXvcsxfZgwGrTDv58/2XXllVmec4dalPQYATAUCFzBFwtYp5QewRMKV5IWNcrVVxfpZ+d+XWnq8++6TM2GVCJ4z7Gxdoc/sj7EU1/tR6LLLcvrKV0ZKHgsAzYTABUyRUjNRpQJYud5ThWajim0aXWrmKsysVqFAVax4P/8z5r+X/7y/rFioyWn++/mBq9Am3gDQzAhcwBQpNRMVlB98Tt6ZV744Pv+19S4SL9dWIexnDJ5rxYqMFi3KTmpyWuj9Hn3U28j7hhsy6uqq28cCgMgRuIApUmomqpRae0+FEXYpsKMjp507s+royBV8vpLPWGhZ0S+m98eRf8wjj6T0yitJzZ7tVvYBAaDBCFxAA1QSnIIzXJUKE6QefFCht+Dp6/O22+nry5U9d7nPWGg2Lr+oP/+YD3wgowMHErr6agrlAcQLlRBAEym19YxbYFKn3FY1Ybb/2bRJE1oslDpnR0dOixZlx7vHb9uW1p13ziq7VU7YLXXKbdezZo23vdHKlVWkTwBooKpnuIwxCUl/JeliScOSbrPWvhx4/tOSPiMpI+kea+23axwrMO0VqnsKdlcvdbz/ff4SnVS6v1Vwa59yrSOCM1zFNpgO+7kKyZ/Ryp9F838GFM0DiJtalhQ/JKnVWnu1MeYqSX8qqVOSjDFLJP2OpMsltUraboz5V2vtcK0DBqazQkXxfg3Xv/+7M6G+Kf/4/PDl98byn5MKh53g1j49PcmCW+8UGp93ruGCtVdhPlcY+UHNn+Ur15sMAJpNLYHrWknfkyRr7TPGmMsDz10p6amxgDVsjHlZ0kWSnqvh/YBpr1DdU3osQ23fntQbb3iBSBoZn/nJbzba0ZHT008nx0NTucaqQflb75QbX6H9Equp5yomP6iNjEz8mVQj//PX2ukfAMKoJXCdIulw4PusMSZlrc0UeO6opPnlTtje3lbDcNBoXL9oPPig1NMjDY/ND6dSKW3dmtK2bVJLS1q33eY9fttt3p+bb5Z6e6X3v1865ZSUBgelSy6Rbr89rfb2tLZu1aTX+tduxQrppz+VVqzwjg3r9tullhZpw4a0nngirU2bvKXKD3+4/Gf7kz/xvv6DP5h8vP+ZJG8ss2dLc+dKS5ZI7e2hhzdB/ucv9POIG3734otrN3PUEriOSAr+LyUxFrYKPdcm6VC5Ew4MHK1hOGik9vY2rl9E7r23VS+9lFZ7e07XXZfVxz7mz/aktXbtqAYGJs7KrF2bHH/u3nvT2rkzrTVrRrV69ZDuuy+pPXtatGKFtHbtiAYGshOuXW9vq958M61t2zJ6z3vCb52zerX3RzrZ/HRkZHR8qbLUZ9u5Mz32dfnjH3oora1b0zp2bFSXXz55eTLMbFXw5zMwkJ30fdzwuxdfXLt4qzQs1xK4npL0nyQ9OFbD9bPAczsk/ZExplXSLEkXSOqt4b2AGaura1R79yZ0/vk5/fEfD2vx4pO3K3Z3p/Xcc0n19SUmdK0v1gur3JKhf1x/vzNpibDSbXvK1XX5xzz7bFYLFoSr73rsMa8P15NPuvr93598fJji/KibwwJAIbUEroclvdcY87QkR9InjDGfk/SytfZbxpg/l/Qjea0nPm+tLf1PVwAFdXZmdeLEiF57LaGhoZPB5+SWONmiPbTKbRtU6L0kaePGlkmF82FqwYKPhQk/fX0JHTuW1KpV4WbSrrkmqyNHHN18c7hmqgDQLKoOXNbanKTfyHv4F4Hn/1bS31Z7fmCmC4aXWbO8x4aGnAlb4qxZM6qOjtx4m4Ziry+172K+YrNgXV2j6u931N/vjJ87P1Bt3Nii3t6U+vsd3XHHyPjrSkmlwvfUKlc0z2wVgGZFp3lgilR6N1ww0Hz841555NBQodYM5V9fqrdV/mOFZon85yWptzel7m634n0dC3n22aQymYSefbb42IKeeiqpV15J6nvfS+kzn8lMej7MOQCgEQhcwBQJ2/zTFww0ra3eY0NDTuhZnGKBqNA4go9t3jyk555L6s47Z+m55zK6556RSbNqwXqxoDvuGJkQxspZsCCnY8cSWrAgV3RsQZdemtPwcFbr1hU/f6U/ZwCYCgQuYIpUWl8UDDTPPOPVaA3VoRKy0DjyH9uyJaXBwaTuv9+rs/I71nvLl8XbvFe6pHf33SMFZ8uK/YzOOy+nM85w9aEPlS/ap44LQDMhcAFTpJb6olmzvDsTKwlc/kxPf78zHmry72Ls6Ulq61Zp7VpNaKC6bl1GW7Z4fa/8jvWbNw+VbXBaTLllzGI1ZsFjPvjBrIaGvBbz/oxfIdRxAWhGBC4gBoJLimGVavHg80KZ19Mr+Nw994zoiiuy2rixRQsWZELPQBXS0+MtT+bfSRlm6S94zPvel1Uu5xXMp/ibC0DMsAUs0ER6epJav75VPT3JCY+3tlY+w9XZmdXmzUO6444RrViRGb+7MKira1Tvf3/hAOXfrbh4sTupDYSkguMspLs7rcHBpBYtyk54n46OnBYtyo4tVxbW1TU6XjPmd9r3fxZBxX5uANAs+Hci0ETyN6D2A86qVV4oCc5whb0br7Mzq+5uV9u2pdXd7YZebsufzQq2fFi82A29P2Oxuyr7+hIaHEyqr6944AouD3p7SBZeTqRQHkCzI3ABTSQYToIh4rrrvKmt4AxXJSEjv4fWxGW9yUuK/jmLnbdUGOvsPFH2PD093ibc+c1VS/E/++zZk2e4KJQH0OwIXEATKbYtT6EarmDIKDfbVWyWq6trVC0t3j6C5QRbPuSP89Chif8tJVjTVWyLoUK+852ktmxJ6b3vndx/i0J5AM2OwAU0qWCIcF3JcaThYSmXkxKJic+HuXuw0CxQZ2dWt92mgps254e4/Lsbg88tWCDt3SstWFD+cxWr6Srn29/29lF85pnCNVw0OwXQzAhcQAw4jjRrlresNjQkzZkz8fkwS2rlZoHyQ0upZcL85cxKGp6G7ZSf78Ybs3rzzYRuvHHyDFeY5VVCGYBGInABDVRJCGhtdTU05BQMXH6YuuuuFt155yytW+d1iK+EH1p27kxIGh5//NAhR+vXt6qjI6enn/buArzmmqykk9sAVbKkV+3y3+WX55TNZnTddZOL7MMEzlIBEgCiRuACGqiS/Q79Oq4TJxxJk5fVpJMd4rdsUUWByy9inzcvq8FB7739WSu/j9fOndnxXlqLF7sTGqVOBb9o3m8CG0QNF4BmRx8uoIGCfaaC/CDmbxgtSaec4gWNb36zeM+pdesymjcvp9mz3Yp6Uvk9t84+252wV6Lfx2vNmlGtW5fRihWZiu4srES5XlqHDnk3DMyfXzhsluN/jjvuqGzmDwDqgRkuoIGKzcwUWiJbsMALGo88ktZPfpIaf31wNuyee0bU15eoueeWH/Tyi+XLqaVOqlwdlh+4/J9DpZgFA9BIBC6gCRUKB6ee6gWNVauyWrjQnRCOgkGlmp5Uld7xWEwtDUhLjTuTkY4ccZRISPPnV3RaAGgKBC4gJvyZnfPPd/WFL5ysn8oPKrXO5NTSRLSS1/b0JLVxY4skb7mv1LgPH3bkut5yYjLkSil3JQJoJgQuICYWLvQC18GDEx+v91JZsAt98HtfqSBTyVj8ujHv69LLn35D1TDLif74+vud8fMTuAA0GkXzQEzMmyel09Lx407oTazzC9HDbvJcqGjft3Fji7ZtS4/PTlWroyOnefNyevvbJxfh54/z4EGvfssPnWHGLqngDQmlsAk2gKgwwwXEhON4S2pvvuno0CFHS5ZUFj687X3C1VhVuqxYzfJdX19Cx44ltGrV5Fmx/HFWEriqbaxa6H0BoF4IXECMLFzoBa6DB8MFrvzgFDZI5S8NBgNVflf5np6kPvvZWTp2LFlRU9FSY8l/rpLAVcsSK5tgA4iK47rV3WIdAXdg4Gijx4Aqtbe3iesXvSeeSOq555J697uzuvrq+szAtLe36b77jpecofLvXFyzZnRSw9Mbb5w9Xiu1YkVGjz9e/y7u996b1qFDjj71qVFt356gGD6A37344trFW3t7m1PJ8dRwAU2kXA2R3xrCn/Gpl1I1W1LxBq1B8+blxpuKVls7Vkg2692l6C+plhtrIdRmAWg0lhSBJhKsIfK/D87k+Hfp+Xft1Uu5pbTgMl1+vVZwiTF4h2M1tWOFHD6s8ZYQqZQ3xv5+R/39jnp6kqHOR20WgEYjcAFNJBh8CoWEk60h6jfD9eCDk4NdvmDIyh9XoZop/3N0dOTGN74ObnZdCb/D/O7d3ibaXV2jWrzYraibPrVZABqNwAU0kVLhRZLa2qRUSvrJT5L69V9P6iMfqb2OadMmlZ39CYasMOHF/xwnu9ZPrv0KKnWX48GDjqxN6Ac/SOjo0WToMRQaDwA0CoELaFKFQsK3vpXU1q0pHTzoaGAgoUSi9iWy88+XduzIjs1CFZbfaiHMTFihbYaKBatSS36HDjnq7fXC1qJF2bJjAIBmROACYqS7O61du5Jqb89p1arSS3SFts7Jf767O63BQWlwMKm+vuKBK2zAKbfcWCxYlZqx+uUvHa1YkdPixTl96lPcmQggnghcQIx0dY1qcNDRmWe6+vSnR3XVVcXDR7mtc/zwc8kllXdkLzW+4H/DPl8q0B065MiYnL70pVG97W1N08YGACpC4AKaXP4y3LJlw3r00VTZwnn/bj7/6+LPp8bPXeuGz+VmwipdCgy2hAizjyIANCsCF9DEenqSuvPOWRoc9IrFOzuz43cqvvlm6cDlhZviTUi9dg2utm3zZrtqbd8QhYMHHeVyJ1tCAEBc8VcY0MS8GquTxeKStHixK8eR3njDUSajmoJIV9eoWlrSWru2sq1/psr+/V6oPP10ZrcAxBuBC2hihTZibm2VFi3y9lTs73d0xhnlw0ixpcLOzqxuu00aGGj8bFYhr7/uB67iBf0AEAcELqCJFat5Wro0pzffTGrfvnCBK+xSYbMtKe7b5+0+tnQpM1wA4o3ABcTQ0qWuXnhB2r8/Ian87E/YpcJmWlIcGpIGBx0lk94yKgDEGZtXAzHk1zTt2+ctuZXbnLmzM6vNm4cmzFr5r3nwwdLHlRLlptAHDjhyXem00yiYBxB/BC4ght72NlezZklHjjg6evTkUmB3d7r8i8f4r9m0qfpx+OfYuLGlpuBVKLh5s3fSGWdQvwUg/ghcQAw5zslC8n37EurqGq24ean/mg0bJj8XdubKP4ek0IGv0LkLBcaTBfMsJwKIPybqgRgJ3m24dKmrV1/1lhWr2VvQf017e1oDAxOfC1s875/DG5cbKvAVOnd+7ZjrBgvmmeECEH8ELiBGgmHlj/5oWFJyvFdVPVVaPF9J4Ct07vzXHzzoFc3PnevqlFPCjhoAmheBC4iRYFhZssRbajtwIKFsVkrWsW69mhmzep472A7CqX+eBIApRw0XECPBuwjnzJFOPdVVJiMNDMQrlZSrEaPDPIDphsAFxJjfEHTPnngFrnJ3Ve7Zwx2KAKYXAhcQY8uWeYHklVei+VWOos9WT09S/f2OVqzIqKMjN+n8R496G3O3tNBhHsD0QeACYuzss73AtXdvQqMRNIevpr9XmHP29qa0eLGrvr7EpPP74fGss3J1rUsDgEaiaB6IsblzpSVLXB044GjPHkfnnOPNCBXbrLpSUWz1U+icwa9ffdULXMuXs5wIYPogcAExt3x5TgcOJPXqqwmdc44XrsL00fJD2e23S6tXFz53FHcr5p8z+HUudzJwvfiioy9/ubXm0AgAzaCqwGWMmS/pnySdIqlF0uestT/OO+bPJb1L0tGxhzqttYdrGCuAApYvz+nHP06OLcUVbiRaiB/KWlqKB66ptn+/o6Eh7+7LRx6ZHBrrNXMHAFOt2hmuz0naZq3938YYI2mzpMvyjrlM0k3W2jdrGSCA4np6ktq8Oa35872lxMOHpfnzw81M+WFsw4b61WdVIxiiFi3yHjv77Jy6urwlxWBoDNsBHwCaTbWB688kDQfOMRR80hiTkHSepHuNMYslfd1a+3flTtre3lblcNAMuH5Tb+tW6fHHpUsukS66SDp4cJbOPTfca2+7zfvjady127pV2rZNamlJ69Zbvbq0lSul88/3x3cyEN5+u9TS4oXE9vbGBsVmwu9efHHtZo6ygcsY8ylJn817+BPW2ueMMUvkLS3+bt7zcyV9TdJXJSUlPWGM+b/W2hdKvdfAwNFST6OJtbe3cf0aYO3apEZG0rrmmozeeiuhZ5/N6eyzMxWdo9HXzv8M73vfqHbtSiqdlubOHZm0v6PkLX36y5+Fnp+JGn39UD2uXbxVGpbLBi5r7dclfT3/cWPMOyV1S/rv1ton854+Lmmjtfb42LGPS7pYUsnABaAy/tLhiRPSX/5li/bsSej4cWnOnEaPLDz/Mzz3XEJ79kjnnJNTS0ujRwUA9VVVHy5jzIWS/lnSR6y13y1wyPmSthtjksaYtKRrJf2k+mECzS+KJqFhzZ7tNUHN5aSXX45ne71du7yfmzG0gwAw/VT7N/OfSGqVtNEY8wNjTI8kGWM+Z4z5oLX2RUnfkPSMpCcl/YO19ud1GTHQpKJoElqJX/kVL6i8+GL8Atfhw9LrrztKp70ZLgCYbqoqmrfWdhZ5/KuBr78s6ctVjguInSiahFbi3HO9zuyvvZbQsWPSvHkNGUZVXnzRm90691yWEwFMTzQ+BeokiiahlZg925sdeumlhF58MaErrmjumaJgO4jBQW9W7oILaPUAYHqK39oDgKIuvNALWT//eVJuk+/77C/B/v3fp/Xmm45aW6Xly5t80ABQJWa4gGmkoyOn2bOlN95wtH+/o6VLmzfA+EuvF1zghcR3vCPLZtUApi1muIAmUuudjqmU9M53estyP/nJ5HM08k7KfJ2dWf2f/zOktrFWNpdc0txLoABQCwIX0ETqcafjJZdk5TiStQm99Vb9z19Pvb0JjY56LS0WLWre2TgAqBWBC2giXV2jWrNmtKY7HRcs8Irns1npZz+bOJNVj/PXIjjD5rrSzp3e+C69lNktANMbNVxAE6nXnY6XXppVX19CO3cmdOWVWSUS9T1/tYKbT190UU6//KWjU05xde655QNX8K5GNq4GEDfMcAHT0PLlrhYudHXkiNNUneeDM2x+jdnFF+eUSJSvL2u25VAAqETz/E0MoG4cR1q50psFeuqp5mkR0dmZ1ebNQ7ryypz6+hJKp6WLLvLGWS5QNXo5FABqQeACYijM3YYXXZRTW5urgQFH1jbXr/r27d64L7ssq7lzvcfKBSo/rLGcCCCOmutvYQChhFleS6Wkq68+OcuVa5K69L17Hb3ySkKzZklXXHEyPBGoAExnBC4ghsIur73znTnNn+9qcNBpmk2t/dmtlSuzmjOnwYMpo5n6lgGIt+b4GxhARcLOBiWT0jXXeMc8/XRS2Ygnj8oFlFdfdfTaawm1tkqXX978M1kU6gOoFwIXMM294x1eU9GDBx3t2DE5CD34oOo2i1MqoGQy0rZtXieaK6/MqrW15reLHIX6AOqFPlzANJdISO95T0YPPJDW008ndd55Ob3tbSdvW9y0SeO9sWqtn/KDSaGA8uMfJzU46GjRIjcWs1tS4/uWAZg+CFzADLBsmauLLsrqhReSevTRpNavz8hxvOc2bJBGRuozi1MsoLzxhqNnn/Vm0G66KaMUf/MAmGFYUgRmiOuvz2ruXFd793od6H0f/rDqendgfh1XLid9//sp5XJeB/y3v71JmoIBwBQicAEzxOzZ0nve44WqJ59M6Y03nEjeJ7+O6+mnk9q/31Fbm6vrrmN5DsDMROACZpDzz8/pwgtzGhmRHn44pbfeqv97BAvNf/GLhJ5+OinHkd73voxmzar/+wFAHBC4gBnEcbwaqtNPd3X4sKNvfStV91YRfsuKq67K6bvf9Yq1Vq/OaPlylhIBzFwELmCGSaelW28d1bx5rvbsSejb31bd91o8ckTaujWl0VFvr8SVK5ukzT0ANAiBC5iB5s2T1q717hZ8/nnp299OKZOpz7kHBhx94xtpHT3q6O1vz+m9782O3xEJADMVgQuYoZYscbV27ahmzZJefDGhhx5KaWiotnP+x3842rzZC1tnnpnTrbdmlKyynyrb6gCYTghcwAx29tmuPvEJae5cV6+9ltA3vpHW3r2VT0dls9KOHQk99FBaQ0OSMTn92q9lNHt29WNjWx0A0wntB4EZbskS6aMfHdVDD6U1OOjo/vvTuvDCnK6/PqO2tvKv373b0RNPpDQ46AW1lSuzWr06q0SN/5wr1bUeAOLGcetdLVs9d2DgaKPHgCq1t7eJ6xdP/rUbGZF27Ehqx46kMhmvuH7ZspzOOcf709bm3eWYy0n79zvavTuh3bsT6u/3gtapp7q68caMzjmnaf5OmRH43Ysvrl28tbe3VbQcwAwXAElSS4t07bVZrViR1Q9+kNKuXQm9/LL3R/LCVioljeZNOM2aJV19dUYrV+aqrtcCgOmOwAVgggULpA99KKOjRzU+i/XaawkND58MWwsWuOroyGn58pzOPNNVmjIrACiJwAWgoLY26eKLc7r4Yq+HVjbr/UmnRZsHAKgQgQtAKMmkWDIEgCrRFgIAACBiBC4AAICIEbgAAAAiRuACAACIGIELAAAgYgQuAACAiBG4AAAAIkbgAgAAiBiBCwAAIGIELgAAgIgRuAAAACJG4AIAAIgYgQsAACBiBC4AAICIEbgAAAAiRuACAACIWKqaFxljHEl7Jb009tCPrbV/kHfM3ZI+ICkj6XettTtqGSgAAEBcVRW4JHVI+om19j8VetIYc5mk6yWtknSmpC2SrqjyvQAAAGKt2iXFlZLOMMY8YYz5jjHG5D1/raRHrbWutfY1SSljTHtNIwUAAIipsjNcxphPSfps3sO/JelPrLX/bIy5VtI/aeIM1imSBgPfH5U0X9JAibdy2tvbQg0azYnrF19cu3jj+sUX127mKBu4rLVfl/T14GPGmDnyarNkrd1ujDnDGONYa92xQ45ICv6vqE3SofoMGQAAIF6qXVK8W9LvSpIx5mJJrwXCliQ9JekmY0zCGHOWpIS19s3ahgoAABBP1RbNf1HSPxlj/LsQN0iSMebLkh6y1u4wxvxI0o/lhbrfqsNYAQAAYslxXbf8UQAAAKgajU8BAAAiRuACAACIGIELAAAgYtUWzdeVMeZWSb9mrf3I2PdXSdooryD/UWvtFxo5PpQWZqsnNB9jTELSX0m6WNKwpNustS83dlQIyxjzvKTDY9++Yq39RCPHg3CMMaskfclae4Mx5lxJmyS5knol/Za1NtfI8aG0vOt3maRHdPL/+/7aWvtAsdc2PHAZYzZKuknSzsDDfyNpnaTdkv7FGHOZtfYnjRgfQim51ROa1ocktVprrx77R86fSups8JgQgjGmVZKstTc0eCiogDHm9yR9TNJbYw99VdJd1tofGGP+Rt7v38ONGh9KK3D9LpP0VWvtn4Z5fTMsKT4t6Tf9b4wxp0iaZa3tG+vt9X1Jaxo1OIRSbqsnNKdrJX1Pkqy1z0i6vLHDQQUuljTHGPOoMebxscCM5tcnaW3g+5WSnhz7+ruS3jPlI0IlCl2/DxhjfmiM+boxpuS2AVMWuIwxnzLG9Ob9uWJs+i3Ym+IUeZ3qff62QGgCha6jpAPytnpaLemP5W31hOZ3ik4uSUlS1hjT8FlvhHJc0lfkrQ78hqRvcO2an7V2i6TRwEPBHVr4/7omV+D67ZD0/1lrr5O3Ind3qddP2S9ooS2CimBboCZW5VZPaE75v2sJa22mUYNB6oojfQAAAQpJREFURXZJennsd2yXMWZQ0umS9jR2WKhQsF6L/6+Ln4ettf41e1jS10od3AxLihNYa49IGjHGdIwVY98k6UcNHhZKK7fVE5rTU5JulsZvVPlZY4eDCnxSXs2djDFL5c1W7m/oiFCN540xN4x9/X7x/3Vx831jzJVjX6+R9G+lDm7WKejfkPQNSUl5dyk+2+DxoLSCWz2h6T0s6b3GmKclOZK4yy0+vi5pkzFmu7ySjE8yOxlL/03S3xpjWiS9KOmhBo8HlflNSX9hjBmRV1pze6mD2doHAAAgYk23pAgAADDdELgAAAAiRuACAACIGIELAAAgYgQuAACAiBG4AAAAIkbgAgAAiNj/D0MFdRo/LBnjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num1, mu1, var1 = nums[0], true_Mu[0], true_Var[0]\n",
    "X1 = np.random.multivariate_normal(mu1, var[0], num1)\n",
    "plot_cluster(X1, [0.5, 0.5], [1, 3], 'b')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二个例子\n",
    "$$\n",
    "u=(5.5 \\quad 2.5), \\Sigma=\\left(\\begin{array}{ll}\n",
    "2 & 0 \\\\\n",
    "0 & 2\n",
    "\\end{array}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAHUCAYAAADvF/aYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXxV9YH///e9WckOISQgWwjhIIIIbq2KG9WOVUsLikg32qHtdJZ26q+/frvo46vfR+cx007HqdPvLD9rZ2yngwhGS+vSEdGxKogooOBySAIYFBKSC9nJfn9/JOd6cnOXc5eT5Cav5+PRB8k92+fcT6733c/ncz4fj9/vFwAAANzjHesCAAAATHQELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXpTvd0TCMyyX92DTNaw3DWCnp95Kqhzb/q2maj9r2nSLpN5JmSGqT9CXTNBuTV2wAAIDU4ShwGYbxXUlfkNQx9NJKSfebpvkPYQ75hqRDpmneaxjGBkl3S/pWooUFAABIRU67FGslrbX9frGkmw3D+KNhGL80DCM/aP+rJP1h6OdnJH0isWICAACkLkctXKZpVhmGMd/20muSHjJN8w3DMH4o6X9L+o5te4GklqGf2yQVRruG3+/3ezweR4UGAAAYYzGFFsdjuII8YZpms/WzpJ8HbW+VZLV65UtqVhQej0eNjW1xFgdjraQkn/pLUdRdaqP+Uhd1l9pKSoI79yKL9ynF/zYM47Khn1dLeiNo+yuSPjX0802SXorzOgAAACkv3haub0j6v4Zh9Eiql/Q1STIM41lJt0j6V0m/MgzjZUk9kjYmoawAAAApyeP3+8e6DBY/Taupi6bx1EXdpTbqL3VRd6mtpCQ/pjFcTHwKAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuS3e6o2EYl0v6sWma1xqGcZGkn0vql9Qt6YumaTYE7X9AUsvQr8dM0/xyksoMAACQUhwFLsMwvivpC5I6hl56QNJfmaZ50DCMr0v6X5Lusu2fLUmmaV6b1NICAACkIKctXLWS1kr6z6HfN5imecp2jq6g/ZdLyjEM49mh7T8wTfPVaBcpKcl3WByMR9Rf6qLuUhv1l7qou8nDUeAyTbPKMIz5tt9PSZJhGFdI+ktJVwcd0inpp5IeklQp6RnDMAzTNPsiXaexsc15yTGulJTkU38pirpLbdRf6qLuUlusYdnxGK5ghmHcIemHkm42TbMxaPMRSTWmafolHTEMwydppqQT8V4PAAAgVcX1lKJhGJ/XYMvWtaZpHg2xy1ck/cPQvrMkFUg6FWI/AACACS/mFi7DMNIk/ZOkOkmPG4YhSS+apvm/DcP4taS7Jf1S0sOGYbwsyS/pK9G6EwEAACYqx4HLNM3jkj429Ou0MPt80fbrxviLBQAAMHEw8SkAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC5Ld7qjYRiXS/qxaZrXGoaxUNLDkvySDkv6C9M0B2z7TpH0G0kzJLVJ+pJpmo3JLDgAAECqcNTCZRjGdyU9JCl76KX7Jd1tmuYqSR5Ja4IO+YakQ0Pbfy3p7uQUFwAAIPU47VKslbTW9vvFkl4c+vkZSZ8I2v8qSX+IsB0AAGDScNSlaJpmlWEY820veUzT9A/93CapMOiQAkktEbaHVFKS72Q3jFPUX+qi7lIb9Ze6qLvJw/EYriADtp/zJTUHbW8dej3c9pAaG9viLA7GWklJPvWXoqi71Eb9pS7qLrXFGpbjfUrxgGEY1w79fJOkl4K2vyLpUxG2AwAATBrxtnD9P5J+YRhGpqR3JT0mSYZhPCvpFkn/KulXhmG8LKlH0sYklBUAACAlefx+f/S9RoefptXURdN46qLuUhv1l7qou9RWUpLviWV/Jj4FAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBl6fEeaBjGJkmbhn7NlnSRpDLTNJuHtv+TpCsltQ3ts8Y0zZa4SwoAAJCi4g5cpmk+LOlhSTIM458l/bsVtoaslPRJ0zSbEikgAABAqku4S9EwjEskXWCa5oO217ySKiU9aBjGK4ZhfCXR6wAAAKSquFu4bH4g6b6g13Il/VzS/ZLSJL1gGMbrpmm+FelEJSX5SSgOxgr1l7qou9RG/aUu6m7ySChwGYZRJGmxaZovBG3qlPSAaZqdQ/s9L2m5pIiBq7GxLdJmjGMlJfnUX4qi7lIb9Ze6qLvUFmtYTrRL8WpJz4V4fZGklw3DSDMMI0PSVZL2J3gtAACAlJRol6Ih6WjgF8O4S1KNaZq/MwzjvyS9KqlX0q9N03w7wWsBAACkJI/f7x/rMlj8NK2mLprGUxd1l9qov9RF3aW2kpJ8Tyz7M/EpAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUACNhRXaU7n1ynHdVVY10UYEJJH+sCAADGj63mFu2q2ylJWlO5boxLA0wcBC4AQMAGY+OwfwEkB4ELABCwpnIdLVuACxjDBQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAuIAJRAHYMS0EALiACUQB2BG4AMAFTCAKwI7ABQAuYAJRAHaM4QIAAHAZgQsAxikG3gMTB4ELAMYpa+D9VnPLWBdl3CCEIlUxhgsAxikG3o/E059IVQQuABinxmLg/Y7qKm01t2iDsXFcBhpCKFIVgQsAkmwsQstvjzymLe/9RrdWrNGq2deqo7dDHb0dGvD3a8A/oAENSJK88srr8Srdm67cjDzlZuQqLzNPuRl5yvRmOmpBcvv+Ip2fpz+RqghcAJBkTru9Yg0uO6qr9Mh7/6VbFtyqFaWX6nRng8592KITTaf0H4cf0tGWWr3felynOk7FVe4Mb4ZKppToguKlWl5ykY6cMVWaW6qCzEJ5PJ6Y7y9edBtiIiJwAUCSOe32chIsevt7Vdd2XMdbjun+N/5e7555R8dajmrdovWSpNzcLHV0d+uC4qXyyKPLyi7X3IJ5Q61XOUr3Zsjr8cqjwcA04B/QgH9AvQM9gVaw9p42dfR2qHegV7Pz52p2/lxJ0m9rBgemZ6dPUWlOqcpyZ2pBUYXWL9rg6P7iRbchJiKP3+8f6zJY/I2NbWNdBsSppCRf1F9qou7GTrgWrtbuFh1tqVVtc43ebz2uvoE+SZJ55l0dbjqkS8su0+p5N6o0p0znz6mQOrOUl5GvjLSMuMvi9/vVM9Cjtp42+c41qaGzXg0d9arvrNe53s5h+2anT1F54QJVFC1UeeECTUmfEtd9TnZ89lJbSUm+J/peHyFwISn4D0fqou7Gh47eDh1qekvv+d7R6c6GYdvKcmeqomihZuWdp9KcMuVk5AS2uV1/fr9f7b1tauhoUF3b+zrWUivfOV9gu8fj0ey8Obpg+lItnrZEmWmZI85x55PrtKtup1bPvUGP3DI20zmMx9DHZy+1xRq46FIEgDHi9/tV1/a+3jx9QNXNR9Q/0C9JykzL1PyC8kALUl5mftRzuRUoPB6P8jMLlJ9ZoIVTKyV9Qme6fDraPNgCd6KtLvC/F+p2aUnxBbpwxgqV5pQGzjEeuggZF4axRuACgFF2ru+cDjW+pbcaD+hM1xlJg8Fm4dRKXTj9Is0vLFe6N7b/PI9moJiWXaxpZcW6pOwydfV1qaa5Wm81HtQHbSd04PR+HTi9X7PyztPyGSt0/rQl4+LJwvEQ+jC50aWIpKBpPHVRd6Onu79b+xte1776verq65Ik5WXma3nJRVo2/UIVZBXGfE6r/uwtXJLGpPussbNRbzbu19tNh9Xd3y1JKsgq0BWzrtLS6RfK62FxEzs+e6mNMVwYE/yHI3VRd+7rG+jTgdNvaO+pV9XZ2yFJmlswTxeXXqqKooVxBRErYH3t0s26rvSmYduSPWYq1u7Knv4emWfe1b6G19TU2ShJmpY9TVect0rnT1sybIqJyYzPXmpjDBcAjBMD/gEdanxTu0++oraeVknSrLzztGr2NZpXMD+hc1tdiJmZ6bruxuGBK9ndZ7F2V2amZWpZyXJdMH2Z3jvzrl758I8603VGT9bu0N5Te7TqvGtUUbSQ4IVJJaHAZRjGAUktQ78eM03zy7ZtX5X0dUl9kn5kmuaTiVwLAFJJQ2eD/nDsKTV01EuSZuSUatXsq7WgMDlBwwpTm5ZvGrEt2WOm4g1wXo9XS4ovkDF1sd72HdIrH76sxs7Terx6uxZOrdQN8z6p/MyCpJUTGM/i7lI0DCNb0h7TNFeE2FYmaaekSyRlS3pZ0iWmaXZHOCVdiimMpvHURd0lV99An/acfEV7T+3RgH9ABVkFumb29Vo87XxXWnQSqb9QXYVuTp/QN9Cng6f365UPX1J3f7ey0rJ07ZzrdWHJRZOytYvPXmobzS7F5ZJyDMN4dug8PzBN89WhbZdJemUoYHUbhlEj6UJJ+yKdsKQk+qPPGL+ov9RF3SXHiZYT2mHuUFNnk3JyMnXpeZdqdflqZaVnuXrdFxqe0cNvPhxo7bJ+Xr90fcTjHn92W6BbcvMVm8K+lkwzS1frykWX6qkjT8n0mXq58Xl92HtMtxq3atqUaUm/3njHZ2/ySCRwdUr6qaSHJFVKesYwDMM0zT5JBfqoq1GS2iRFffyGpJ+6+H9qqYu6S1z/QL9e/OAFvdGwT36/X8VTivXJ+Z/S7Pw5aj3bI6nH8blibWEqKcnXg/se0q66nerpGZyR3vo5eDB9sLXl69XT06e15esDfwOhXks+j1aX3qzzMhZoV91OHf7wPZmnanXNnOu0YsbFMbV2jccJTZ3is5faYg3LiQSuI5JqTNP0SzpiGIZP0kxJJyS1SrKXJF9ScwLXAoBxqb23Xb+v+a1OtNXJ6/Hq8lkf1xWzrop5Hi1LPPNphRpj5WS8VaixXm7MmRUqFHk8Hp1fvETzCubr+brn9I7vsJ57/1md6jilG+f9iaNlinZUV+l7L31Hvi5foOzAeJVI4PqKpGWS/twwjFkabNWylqh/TdLfDI3zypJ0vqTDiRQUAMabU+0n9duax9XW06q8zHytqfiszsufndA5YxmgvqO6So8/u01ry9cPm/7BzeARa4uSPRQ1dNSPODYnI0e3VHxada3H9Ot3HpZ55l35zjXpMwvXRp2XbKu5Rb4un4qzi5nQFONeIoHrl5IeNgzjZUl+DQawbxqGUWOa5u8Mw/gnSS9J8kr6oWmaXYkXFwDGh0ONb2rn+/+tvoE+zc6fo09XfMbREjzRxNLCZLWG9fT0jVrrTqwtcPZQJCnssS99+EcdbalVmiddxrTz9at3/kNrKj6ruQXzwp7bHk5p3cJ4F3fgMk2zR1Lw/6XYbdv+C0m/iPf8ADAeDfgH9HzdTu1veEOStGLGSl0/9waledNGvSwbjI3KzEzX2vLhg+PdHNcU6xQRwfvbZ8MPtd/ahbfJ75GOtxzTNvMRXT/3E1pZeknIc4+HJYMAp5hpHknB4M/URd051z/Qr6eP/V7v+t5RmjdNN8z7pC4suSip14hn0Hzw0j5WK1SyZpofbQP+Ab30wYvae2rP4At+v/affkN3Lv7chApYfPZSGzPNA4ALevt7taP2cR1trlVWWpbWLrpdc/LnJv068S5CbT8u1Rdq9nq8umbOdZqWPU1/OP60HjuyTcdaj0piYDxSF4ELAKLo7e/VEzWP6XjLMU3JyNHti+5QWe5MV65VUVihg9n7VVFYEdNxweOZkhlMxmrqhWUly5WZlqUjZ96TJC0tXiq/3z8pJ0lF6iNwAUAEfQN9gbCVk5GrO4yNKskpce16tS218nX5VNtSG9Nxbo5nirfVLRmMaYv1/cvv0RM1Veof6NfzdTt1/dwbCF1IObEvUQ8Ak8SAf0C/q31i1MKWNNhCtXruDSO6A3dUV+nOJ9dpR3Vyx2Q5OW+4MsV6vnjvYUHRQn124TqledP0RsPreunDF2M6HhgPGDSPpGDwZ+qi7sJ7oW6X9tXvVXb6FE3NKtLTx54csykI7nxynXbV7dTS4mUqzS0LlCPR+rPOG8sA+0hdjNc/eqUO+w5pafEyPX/HKwlfy662uVpPVFdpwD+gTy24VUunL4v5HOMJn73UFuugeVq4ACCEtxoPal/9Xnk9Xn1m4Vo9fexJ7arbqa3mljEpj9XKJGlEORJp/XLaemW/htXFGOt7EUtLWSgVRZWB9+C/jz+tD9pOxHUeYCyk3XvvvWNdBsu9nZ3O1xvD+JKbmyXqLzVRdyO933pcv6/9rfzy65PlN2nRtMXK8KSrs69TG4yNWly8JOFr7Kiu0n177lGGJ93R+RYXL9Fti+5QUVbRsHLk5mbpu899V7vqdqqzr1O3LbojpnJY57WX4e6Xvqu/2PU11bef1PXzBgPOfXvu0a66nTrWclQdve2akVOqzcu+NqLsVvlCbQt1rVjNzJulrr5z+rD9Q9W01MiYaig7fUrM7+d4wGcvteXmZt0Xy/60cAGAzZkun3bUPKEB/4AuLbs8MM/Wmsp1euSWqqR1J8baSmS1MEkaUY54xn1F2lZVvV2+Lp+qqrePuIYkHfYdUmlumdZUDh5//aNX6vpHr9SO6qqkv0+hynvd3E+ovHCBzvV2qqp6u7r6ukK+n26NewPiQQsXkoL/p5a6JnPdBbeK9Pb36lFzi9p6WlVRtFB/Un6za0/DWS1mFYUV+sWhf4vaMmO1MHX2dSrDkx4o98o5F2lOdkXIliP7McEtX9a23Sdf1py8OcOOrW8/qeOtx7Su8vZAC1e4Frb79tyjvfV7dPrc6RFlS2ZLk/1ebjc2qKJooWqaq+U716QzXT4tLV42ogUy0v2PB5P5szcRxNrCReBCUvAfjtQ1mesu+Av5xQ+eV21zjYqnTNdtxh3K8GbEfe7gMBf8uxVgfnHo34aVIVzXmL1L02rN6ezr1BdXfGFE/VnnqCisUFH21JDdoBmedO0++bJ8Xb4RgeT6eTfoL1f8tdp6WkeUJbhb8GDDGzLPvKcZOaX6y4u+Oaxs8YacUO9BcJduujddCwoX6G3fYZ3ubNBlMz+mP1/xzbDv2XjsZpzMn72JINbAxTxcACYt+2Sh77ce1+v1++T1eHXzgluVlZaV0LmD564KN5dVqLUGQ+0Xap6tcIPPA+eYq7BPA1rnCre2YaSy2NW21Kq9r12XT/t4yPuKh3Xdho76YU9DBpehKHuqvPKo6sg21TZX6yfX/KMKs4qG3SMz02O8oIULScH/U0tdk7nurNaa8qIKPXbkUXX3d+uK867SkuILEj53cOtKuC7E4BYjJ60y9mNC1Z/T7spog9idlCV4H+uc5pl34+5atM7Z0duuvfV7IraW3f/63+tA4xvq7uvW9JwSXVC8LGUmRZ3Mn72JINYWLubhQlIwn0zqGk91N1ZLyDxz7CkdanxTZbkz9bnzv6g0b5pr13IyF1Us70Ok+kv2HFuxSHTOLadl2VFdpf969z81fcp0zS9coNXzbtDFpZfGXe7RNJ4+e4gd83ABSFnxzu+UiNrmah1qfFPp3nR5JX3+6fWuPdW2o7pKDR31Wlq8LGKXW7xPMAaXe4OxUUuLl6mho97xPcVTB6Gun+icW5KzJ0PXVK7Ttk//Vt++5LuSpBdPvKAzXb64rwm4hcAFYNxIxpe0XbRpAQbX5ntOkrRq9jV6yuXJTbeaW4ZNqRCuvBWFFVo99wZVFFY4mtYgXEhaU7lOpbllOuw75Pie4qkD+/UjTV9hcWO6hsqpi3TB9GXqG+jTiydeSNp5gWRh0DyAcSPZg5yjDfp+s/GAznadVfGUYq2cccmIAezJFu38wYPdrW65cOV3ct5Y7ymeOqgorNDB7P2qKKxwNNA+3KD4RF0z5zpVnzVVffaITrTVaU7+3ITPCSQLg+aRFAz+TF0Tue4iDfru7u/Wb2seV99Ar/6k/GZNzylJykzokQSfP3j6gwxPemAm96KsIi2dPnJuKbsd1VW6+48/0PlTL9APP35vyPOuqVwX8p6SOTP7Lw79m972HQ5MQRGtzLvqdmpGTqkkRR0UH0s5M9My1e/v14m2Ovm6mnTh9OXjegD9RP7sTQZMCwEAQyK11rx26lWd6+3U7Pw5WlhUmfRrOxnwHdwaZJ8+Yqu5JWSXnP281r49PX2B89lft87r5NqJsLeiRWshe2D//YHFrb+18q6I01LEU85Lyy7XwdMHdKr9pMyz72nxtPNjvBvAHQQuAONesp9ebOtp1ev1r0mSrp1zvSutIKGCQvB9BHf3ORlUb533xQ9e0DXnXaubFt6kteXrh13PSTdiMrtP7SErlrpy0n0Zazkz0zJ15XlX6dnjf9BLH/yPKosWufrUKeAUXYpICprGU1cq1F2yl2h58cTzOtn+oYxpi3Vp2eVJKOFHIs3yHnwfwV2M1jI5y0qW64cfvzfk+TM86Xrm+FPqG+hTa0+L3vzGm5qTXTGs+zRcN6JdtO5NJ/do39d6bVfdzojdhEVZRTrWcjTwc7TrR+rmDXfMjJxSmWff1ZmuM3qj/jU9sP/+cbmodSp89hAeXYoAJpxktsac6zunt5sOS5KuPO/qhM8XLNIs79Huw8l9rqlcp331e1VVvV3rKm8f9noirX+xdN2F2tfqKpydNyfqU46nOk7K1+XTVnNLYAHs7730HfmGpnNweh/hyuz1eHXleVfrdzVP6LHqbTpy1kz6AP1YjdUccxg/CFwAxr1kPr14qPEt9Q706sw5n/5q19eT/gUYKjTZv2wjTQLq9D5/tOon+tGqnwx7LdEv9FhCbaR9m7vORh235uvyKd2brorCimGvFWcXxxSqI5WjsmiR8jMLZExdrOLs6WrraXUcKN0IR8kcM4fUROACMGkM+Ad08PQbkqR3zrytV0/tlhTbF2C0L+NQoWk0vmzDXSO4vOHKH0uoDbXvt1beFWilslqu7KzrWlNI+Lp8qm2plTRy0L1Tkcqc5k3TRTNWqK2nVTdPrVSavFEH6FvcqC+3pxzB+EfgAjBpHG2uVXN3s4qyivTlC/5UuRm5MX8BxvNlHM+XbaytLPZrWMfK79eLH/6P+gb6AuUNLr/T5XPsYeWB/fdLGgxZ1jHRFsO2d7X+3aqfDtvPfqx9/0RbmC4suUi7T76s2uYafXXZn7laX9GwkDYIXADGrWR37ew//bok6aIZF+uymZfrs4tuH7bdyfXi+TKO58s21mBnDy0NHfU67DukdG+6+gb6lO5ND5TXPkmp0+vY95Gkw75DgdedtpJFmzrCfg3zzHv6oP2EzDPvBbbF8zeQm5GrxdOW6O2mQzpwer+um7va0XGEI7iBwAVg3Epm105z11kdbzmmDG+GlpVcGPf1gr+M3RoMHU+ws8q/tHiZVs+9QfL7dbDxgNZV3h4oW21LbaA7z+najsFlaeioj7ls0UKM/RrffuGvJA3WWXDYi/W9XjnjYr3ddEiHmt7S1bOvZYoIjBkCF4BxK5ldO0fOHpEkLZxaqSnpU2K6XqRQ5db4LCetLDuqq/T4s9u0tnz9iHm9rO7Chs4G7T75inZUV43Yx1rbcfXcG6IuEB3ckpVs9mvYn8K8tOxyNXTUq6GjPvAkZCxlmJk3S8VTpst3rkkn2uo0v7A86WUHnCBwARi3ktm1U9M8GLgqpxoxXy9SqAo1dmq0Hv23zzRvld1+XStQWT+H2sd+D+HEOs7Lvs/dL31XVdXbdVHJCsnjcfTeBD+FGdxyF2sAr5y6SL5zTappPkLgwpghcAGY8Dp6O/Rh+wdK86apvHBBzMdHammzB5jgxabDhZBEgpn92A3GRmVmpmtt+foR+9390nf18ocvKU1pmpk3K2rZIwk3a7598Hy4UFpVvV2+Lt+IwfuxiPcpRktl0SK9enK3qs9Wa/XcG+NeWYC5tJAIAheACa+2uVp+v1/zCucrKy0r5uOdBpMNxsZA95f15RwqqISb5NNqDVpXefuIebYsoZbwCaWqejdc/SMAACAASURBVLu6+7skSca0xZIGA2G4sGCftqG2pXZ4t2RHvWbnzQncl/W0o731LNz8Y1PSc5SX3q3LZ35M8nhUUVgRsRyhJNrSWZY7U/mZBWrraVVDZ73KcmfGdR7m0kIivGNdAABwW/XQ+K3KokWO9t9RXaU7nxwMG7FYU7lOpbllOuw7FAghS4uXBYKKFH6Szx3VVfr3tx+Sr8unqurtYa+xwdgY6Fbbam7RMzXPDJtOwbKu8nblpedpdt6cYQtah9rXKteuup3a8u5vtKtuZ6D1ygpW5/o6A/dllWNp8bLAgPs1letGLLa91dyiD9pP6PJZH9cjtz6uR26pUm1LbcRyuMHj8Whh0UJJH/0thBKt3u3vPRArWrgATGi9/b16v/W4PB6PKqZWOjrGaUtGqC6m4O4ve9AJNbDdOs/3XvpOYAoH+5I9wYJbe+xdivbyhJqN3l6+cK+bZ95Te3v7iNftLV+hyhHqvQl+AtLpU5FOxdLFt3DqIh04vV81zdVaNfuakPtEq3emi0AiCFwAkmq8jXM53dmgvoE+Tc8pUV5GnqNjnD4dGeoLOvhLObibMdwcVL4un/LS8zS/sDzigtrB7+/mKzapsbEtbHkskcKCfTzWp8pvjhqswnU/2llPFC4tXjZsjjDrNauFK5G/kVi6+Obkz1WaN01N5xrV09+jzLTMEfswGzzcRJcigKSK1nUVq3i79yz1nackSTNzZzk+JlT3WCgVhRUqzi4OTCJq2VFdpesfvVLXP3qlJAW6GR/Yf3/Ie7G6qoqypwb2Cz6fdZz1/n7vpe8MO481SD7Lm6WKwoqY3jcrCB32HVJtS+2Iew8+l1WGqurtjuva3roladhx8daxvYsv2jnSvemaPqVEfr9fDZ31IfdxWu9APGjhApBUyW4lSHSgcv3QJJ1lOWVJKY+dfRJR6aOWH2umd0nD1g881XFSh32HtPfkHknDW8UkBSb8DBY8UP7g6f2BNQs3X7FJ0vBB8rUttYGxUvbz28sY3A0abjLT4EH+kgLB6YpZVw5rDbOznly0Bslb74l9/Jl1XLx1HOkJ0VDKcmeqoaNe9R2nNCd/ruPrAMlA4AIQk3gWb07knIkGuPqOwRaueJ9MiyS4bPb5oqyWHCtcWF2GktTe1z5iWZyt5ha197WrOLtY31p5V9jr2Lvn7O/JusrbteXd36goe6o2GBu1r37vsCV87NeJ1g0avL99kL99stRwT1Laz3n9o1fqsO+QstKyhw2wj9btGkm0sXPhzMydqTd1IPA3AYwmAheAmLjxaHy8Y4+i6e7v1pkun9K8aSrJmeHomFjGoIUKDta/oY6tKKzQ7pOvDNs3+NiKwoqQizg/cstHXWWh3pPgQfIP7L9fvi5f4HqhyhiNvRvQvlB1uOMjvXfd/V0qzS0LG9KDHy6IJNbQaCkdCt0ELowFAheAmLgxsNitwcoNHfXy+/0qyZmhdK+z/9wFf5mHChHhgkXwl741GL25uzlQnuDgEnysvWtMUtLCrb3M9vAW6f7s3YBOW8OCy/utlXcFxqQ5Wa8x0jxd9sH6mhv738v07OlK96brbNdZnes7F3aJJ8ANBC4AMXHj0Xi3Hrdv7j4rSZo+pcTxMeG6CaWPQoTTVj772n+S9EH7iWEtONG6xkJ1CzptgbPGUEUbJ2WVsaGjfsT9xbqUTrgnMmOp290nXwm7XmLgHuZqWGh0Ks2bpuIp09XQUa/mrrOakkfgwughcAGYsNp7BueTys/Md3xMpG7CSK9FkpWWrZIpJSrKKhp2TKSusR3VVYFlcXaffCXQ6uM07Dm5jx3VVTreckySdLzl2IgFrq1pH6x9nQS9Ux0nAwP6YwlaTkJe8D3EMwVJXkaeGiS197ZH3RdIJgIXgAmrvXdwfiqn82+FEqqFxh6KIi1TY29lCrU9UnCzxmBZA+2Dl/OJtTst1H08sP9+tfe1yyvvsIH8wd2bB0/v18zcWWFbnizWAPt0b7oa2ut1/r+XR1ymyM7Jeon2e4i0RFIkeUPhu72nLfDaeJs7DhMTgQvAhGW1YuTF0MIVi0RnJnfS3Ta/sHxEcAt1TLTJSCOFill558mYtjjkQH5rCoqZubOGtTyFul5FYYVe9L6gvoE+vdf8rvoG+lRVvd1R4HLyXtjvIdwSSdFY4dvewsUaiRgNBC4AE5bVpZhIC1ckbs5MHi1kWeHja5duVmtrZ6C152D2YEBq6KgfFojsc4NZrXOSQj6BaAmegiJU65J1PUvfQJ+Ks4t1UckKHWw8EHGZIvu9WAPrrSkxgsefBXenOmkRCyVU4GKGeYwGAheACattqNsoL9OdwOXWYP9wglt4dtXtlK+7USdaTgRae6yg09zdrMO+Q4EWp+CxUfb5tMLdg70Vy74Uj711aV3l7SMmP430hGGobVZZrJ8lhXxSMzhkxfPeW38L9i7F0a5HTE4ELgATkt/vV2dfhyQpJz13jEsTu1DdXPYnCq2WoJqWI4GxXn+36qeBMLS0eJaau86qva9deel5YefRitSqY5Vhb/oetfe1B2ajDzc3l72sTu7HXpZQM92HCnGJBqPcoRaujt4Oxm5hVMUVuAzDyJD075LmS8qS9CPTNH9n236XpD+V1Dj00tdN0zQTKyoAOOeXX36/X16PV2netJiOHasvYnvX2hWzrhw215T9iULpo1aZG6uu1vst72t+YfmIQGUFtOBt9uMjsa5tnnlP7e2DXXBOWsas8trfQ2t5I/n9Ix40CPdgQqifE5XmGfza6/f3M3YLoyreFq7PS/KZpvkFwzCKJR2Q9Dvb9pWSvmia5huJFhAAogkVkAb8A5Ikr8cb8/msL2JrHFS44OUkmMUS3uxda6W5ZcPmmrKeKLRaqyzfv+r7enDfQ4FwFBxegpcAiqVM9qcxg88TbbyT9R4ePL1f++r3Bqa4ONh4IOYnC5PJ+nvw+wcYu4VRFW/g2i7pMdvvfUHbL5b0fcMwyiQ9ZZrm38Z5HQCIKlRLhRW4PPI4Ooc9VFhfwA0d9RFbQCK1kIRayNpJi1K4RaQt8wvLJSnQSrT5ik26rvSmiPcTbgZ5J2Wy9rGfI9I17PdiPeFoha1wY76CzxfuSctk8HoG/x4G/AOM3cKoiitwmabZLkmGYeRrMHjdHbTLVkn/LKlV0hOGYdximuaT0c5bUuLOo9sYHdRf6kr1uvvapZuVmZmuTcs3Be6lt79XublZyvBmOLq/f656QAcbDsrX3agDf3ZAm6/YpG2Ht+nhNx8edt5o15WkbYe36fuv/L9q6mzSRaUX6aaFN4U9h93mkk3afMWmkNvuufbuQFkefvNh7arbqdfqX9U/v/WAvn/V97V+6fph+z/+7DbtqtupzMz0wDmt1yKVyX7PwecMPq+kEdew30tBQY4efvNhLZq2SEfOHAl7zuBzv5lzQE2dTfJ1N+rxY9uiHhcLb2ePcnOzlD9lyrj4ux8PZcDoiHvQvGEYcyQ9IelfTPOjlVYNw/BI+plpmi1Dvz8laYWkqIGrsbEt2i4Yp0pK8qm/FDUR6u660pt03Y2DrTzWvfQP9Kujo1teT6+j++vr6w/8a+0f6rzRritJD+57SE2dTSrOLtZfXPitQCtKIu+z/Vqt5Z167YPXBrvoGg7qwX0PqbW1c1hr09ry9erp6dPa8vWB61qv2VuOgsv04L6HtKtup3p6+kK2nNnPK2nENUKVeUd1lQ7Xv6PW1s6I74F1bvtUFs/UPBO2LLHaUV2lX7/zsIqyinTFeVeFLctojeGbCJ+9ySzWsBzvoPlSSc9K+kvTNHcFbS6QdNgwjPMldUi6XoMD7AFg1FhjdQb8A/L7/fJ4InctWvNeRVo82alY54iKZYFsafhTi+npaSGX/AnuBrTPcxWpTNHGNQV3wzm5v3iXIwo1diwRW80teunDF1VesEBXzb4m4fICsYi3hesHkqZKuscwjHuGXvuFpFzTNB80DOMHkl6Q1C1pl2maTydeVABwzuPxKCstS9393eru71Z2enbE/YOXs7Fei0esY4OCv+CdLFtjXSO4lSTcMkHWmK3vvfSdsOe0nzfaskUWJ2Ex3DqO0VqRkj3GaoOxUef6zqk4u1hZaVkR9wsuL5CoeMdwfUvStyJs/09J/xlvoQAgUTuqq7T9yFYZUxervbc9auCyjMWXbfA14122Jpo0pTleWNppK0+o/SK1tsV6/mRaU7lOldMW68naHRFXH2AwPdzAxKcAUo6T1pGt5hYdOWuqt79XbT2tmj5luqNzj8WXbfA1gwOYk5amUK1k1ntkdZfK79fBxgOqKKwYcXy4ebNC7WsXKqA6Ca1j1YoUWO7JpdUHgHAIXABSjpPWkQ3GRp3ubND8gvJh6+aNF5HGVdlbiqwpHPae3KMH9t8fcgzWjuqqwOzv9lYyax6sv1v1Uz1yS5Wuf/RK+bp82n3ylRHlCX5Pa1tqA/uGC3zhgq+T0BpvsA0e1xVqGolIgby9d7D7NTeDpwMxughcAFKOk9aRNZXrVJg9Va+delUdPaMfuKK1wgWvHxiuy21p8TIVZxfL1+XTYd8hPbD//rBrEVqzv1sBLC89b1gXYnN3syQF/rULfk+jzUXmZJxZLO+H033swTAQRoeWHrLKESmQd/S6u6A5EA6BC0DKcdo6Yn2pWq0aoylaK1y0SU6Dg8+3X/irQKiQPgonX7t0c8gxYId9h7S0eJlKc8sCrxdlFemD9hMqyioacb1QTx+Gm2XeuoZ9nFmkFjt7OIs0e7/TlkvrX+t6GWmZKk4vDnR/RgrkdClirBC4AExY+ZkFkkK36DiRyHxMTqZXkAZDRrjtoQJJcJdhZma6fnXjo8P2tcZfXTHrSl1adnngGtZYLqfjpsLdv70L0wpXdz65LmyLnT2cSYq4iLX9Xyfvi73btbalNuQ+dme7z0qSCob+NoDRQuACMGGV5pRKkuo76h3NxRUs1ifpggNKsp4EDHUdK/BsWr5pxHZr/FVtS61qW2oD60JarV1OrxWufKEWsI7UYheqBS5UqIp1XFe0Vrhg7T1tau9pU1Zaloqypjq+DpAMBC4AE1ZhVpGy06eos7dDbT2tKsgqjOn4WOePChVQoq03aP0bbj4rq9tMUqAFyfp59dwbtH7pyFneQ5U72rqQTu8/3OuRwlKo7kqn83xJ0VsanQa1+qFAWJpbFnP4BhJF4AIwYXk8HpXllul4yzGd6jjlKHBFa6WK1CoVHESCB5Zbx9vPbZ0j1ISr9oH1S4uXafXcG0JOvxBsX/1eHTw9OKXDj1b9JKZWIPt7EG5KiFhaosKFJet9jDSmK3hf69qRzhtJfecpSVJZ7kxH+wPJROACMKHNzJ2l4y3HVN9xSsa0xVH3dzLY3f6vXajxRfaB5ZGCQ0VhhTR35HxWVjdduKkjQqmq3i5fl09V1dv1o1U/CVk2J+/Bwez9jp9CDCXSk4zWOLPm7uZAqHQyUD64jMHHRQpi9R0ELowdAheACc36crW+bKOJNuFnLMHFHpis3+3/SrbgMBS2rAHu4WZod2Jd5e2qqt6udZW3x9USZJW7ubtZS4tnxbwkj/3ews2Yb40zW1o8S8a0xTENlLfKaP/Xfs1QQczv9+vU0N/ATAIXxgCBC8CEVpZbJmkwcPUP9CvNmxZxf/uA80TZ54Taam7RI7dURQwOyVru5kerfhJo2YpnbUir3Id9h1SUVTQsBEYroz2QRVrE2/4kpVXWWIQLo+GCWHP3WR1seEPvnnlHLV1ndbT1aFxPnwLxInABSBnxtNbkZxaoeMp0+c416URbneYXlkfcP9lLzjiZHsK6l331ex0tpxMLJ0v0hHpfw018Gu1+7IEsVMC02INtItNvBAsXxGqaq3W46ZCOtR7V2a4zCXWVAvEgcAFIGfG2AFVOXSTfuSbVNB+JGrji7caLdD5Jw1qJglmBo6GjPmmtaxYnLXah3tdwUy6Ee38ijUWLtp/9+lZ5kt36VHO2WkunL9OMnFItL1keWAoIGC0ELgApI97Wp8qiRXr15G5Vn63W6rk3ujYlQLQn8qTQgcu+jE/wk4iJijb1RPA+wZwE0GGD4+cOtmyFYx+zFrxfqG7VZLR+dfR26IP2E1oyfan+ZcVfKystK67zAIkgcAFIGfG2PpXlzlReZr7aelrV0Fnv2lNq4YJVtKBovW4twhyrSKEk0tQTkZbjiYU1OD7dmx61O9TpHF7BM+pb+8XCel+umHml/H6/5hbMI2xhzKTde++9Y10Gy72dnT1jXQbEKTc3S9RfahrNuttRXaX79tyjDE+6FhcvGZVrSoPzcTV3nVV9xynlZuZpbsE8V66T4UlXZ1+nNhgbh93f4uIlum3RHWHveXHxEmV40vXzAz/T277D2n3yZc3Jm+PoPcrNzdKXdnxRe+v36FjLUX1p6Z86Lt+fP/dVHfYd0ulzp9XZ16nbFt0R+00PnXf3yZfV0duh7v5u/eH408Pq2F7vayrXRX0vblt0h8wz7+q+PfeoorBCRdlTR7ynTty35x7tqtspX1eTKqcu0mUzL9fek7vH5G8wlEQ/e2P1ecKg3Nys+2LZnxYuAKMmWU/hxWPh1EU6cHq/zDPv6spZq1zpVkxk/Je9lcjX5RuxHmGyy7ejukrHW45JkvLS84ZN1hpqxntr3JU19inUnGDWOLRQE7g6neTUEqnr0Qlr+aMl0y7QwqJKSdLCokr9+LW/GbO/wWQby88TYkcLF5KCFq7UNZp1F64FyOLW/2PfUV2ln73xD+rp71ZmWpZm589RUba7a+nFei/We7N6zicCLTpWK0+kc+TmZimzb4o6+zq1ednXhu0XqQz37blH5tn3VJxdrJ9e87PAF7bVKmRv8bJeO956TG/7DodsDbNa6fae2qMZOaXDymLdW0dvu/bW73HUmhbtbyWa+/bco731ezSvYL4uLr1U8wrma2XpJQmfN5kS/eyNp3uZjGjhAjBuRWsBivT/2BMZPL3V3KLnTzynlTMu1rzCch04/UbUpxUTFWvrQ6j3xukcWuHeV6fLEEUbbxY8xizcWLQH9t+vw75DykvPC1m+aMsLxbr4d6RzVBRWyD/Hr5KcEknSxaWXDivLRDCR7mUyIHABGDciDS5PpPvEOt9nKtbqg44PVNNcrZbuZhVmFSVY4uHCTfoZr1gXz3ZyvMXp04fWtWLp1mvvaw/ZJZpI4HbK3hX5N6t+rKoj21WYVagFRcmb2wyIB4ELwLgR6Qs5kQBjP++Ttb/TO77DOnB6v66dc338hQ3B6aSfToV6P4LHQ33t0s26rvQmx8dHEzwnVizzY31r5V2Bpx7jqadkh9T9DW9IklbMuERejzfucwLJQOACkBKS1X2ysvRiveM7rLca39SVs1YpIy0jCaUb5CQwBLdQRRuQHu4a1uD0zMx0XXfjR4ErUguYk2uF61J00vqUaB0lo46tc/jO+fTLQ/+fMrwZWlZyYULnBJKBwAVgUpmVd55m5s3SqfaTeqvpYGBsTzJECwzDJgjV8LUJD2bvH7HcTKjwFDweatPyTcP2ixSMIl0r1D3sqP6oGzHZSx657bX6VyVJ5xdfoCnpU8a4NM4lc5kjjC8ELgCTzsdnXqnHq7dr98lXdEHxMmWnZ0c9JhlfhNbUD8XZxSMCTKgB6dFmXV9TuU4lJfn6xH/cGNgvUjCyX+vpY09p78k9uvul74ZcPNoeDhs66lWaW5aUEDAagaKxs1GHm96S1+PVZTMvd+UabmGqh4mLwAUgpcXzBV5RtFBz8ufqRFudXqt/VVfPvjbqMcl4gjLUk4GxjFuzngJs6KgPBLDHn902bF3CSOezb6uq3q72vnZVVW8PGbjs4VAKP5Yr1vd/NALF/a//WM/XPadbF3xa04bKnypSrSURzhG4AKS0eL7APR6Prp1zvf7znYf1ev1rWjFjpfIzCyIek4wnKGMdo+T0qb7Vc29w/BShFZAuKlmhg40HtK7y9pD7Bd+vNfYrXJeo9Xs0bgeK91uP63e1v1XjudN6rm6n7r7i/7hyHbcw1cPExWMbAFLaBmNjXAs+z8ybpcXTzlffQJ9e/vClqPuvqVwX9snDeMqwo7pKdz65btg4qWi+tfIuLS1eFjh+g7FRNy28KabrBgKSx6N3v3IsZOtWMOvea1tqQ3aJhrv3UPdonUtSYFs870Uofr9f/3Pi+cDvPJmI8YQWLgApLZEWgVWzr1F18xEdbnpLK0svUWlOadLKEO1pwW+/8Fdq72sPdA8GHxfqSUJ7i9JWc4seuaVKm6/YpMbGNsdlddrCFKrlKtYuUScD+C3J6GZ8x/e2Gjrqde3s69TU1aSNiz8f97mAZCNwAZi0pmZP04oZK/V6/T794dhT+vz5X1KaNy2hc1qBqaGjXod9hySFDhvtfe0hj4/2JKGTyVCj/T5aXX9OBvCHmtE+Hu297Xr+xHOSpD+76C+1rGT5sO08/YexRuACMKlded7VqjlbrYaOeu059YquOu/qhM5nBaalxcvCdrVtMDaqoaNe0mA3YfA26aOnFisKK3Tnk+uGhSX7tA2PP7tNJ85+MCzcBbcsxTPOLVQwCx60H885wm1LJAT5/X49e/wZnevt1PzCci2dPnLeLZ7+w1gjcAGY1LLSsnTTglu09b3/0qsnd6uicKFm5s2K+3zh1ii0iyWIRFpPMTjcWeHM/tRicJnitaO6SsdbjsV9vJsON72lmrPVyk7P1k3lN8vj8YzYh6f/MNYIXAAmvTn5c3VJ6WXaV79XTx/7vb645CshZ6B30i0VLkzF0qXldE3GDcZGZWama235eq2pXPdROJurYU8txjrGLBSrG7Q4u3hEq9xYaulu1vN1g12J18+9IezTpjz9h7FG4AIwISQ6RmfV7Gt0tKVWvnNN+uMHL2j1vBtH7JNIt1Qsxzpdk3FN5bphg+ajteI4nZE+FCctd6PBfg+fXrhWzxx7St393Vo01dAFxUvHrFxANAQuABNComN00r3punnBrfrNO7/SGw2va1bebJ1fvGTYPol0S8VybLzXcTpvl9NrhBpsb03hMFbBy34PU6cUq671feVk5OqG+X8SsisRGC/S7r333rEug+Xezs6esS4D4pSbmyXqLzVNlLrL8KSrs69TG4yNWhwUlJzKy8xXZlqmjrUc1dGWGs0vKFd+Zn5g++LiJbpt0R1xnd9+7I7qKt235x5leNJDniuW69jrL9p57e/Rmsp1Ua9x3557tKtupzr7OnXbojvCvpZswfdh/33p9GXq7OvUlbOu0rGWY/J6vFqzcK1mxDmlx1iaKJ+9ySo3N+u+WPanhQvAhJCsMToXl14qX5dPb54+oMerH9MXlnxJBVmFSSjhR5y2xjnpJt12eJse3PeQo27CWN+jSFM3uDn4PNJTlo/cUqWLyy7To+YWSdLquTeovHCBa2UBkoXABQA2Ho9Hn5h7o852nVFd6/t6vPoxbTz/C8pMy0zaNWKdfPTg6f2SQoeoh998WLvqdurFD17QNeddG9es++GECmijMfg83NOVG4yNau46qydqqtQ/0K+VpRdrRenFrpYFSBa6FJEUNI2nrolad9G61yLxeryqKKpUzdkjajrXpKauRhnTzk/aGCGnXYYZnnTtPvmyfF2+sF14RXl5+q35W/UN9Km1p0Uv3LFb5pl34753u0Tew0QEvz/W7/MLF2jbka1q7W5ReeEC3bzg0yk9bmuifvYmi1i7FFloCsCEZF8CJx5T0qdo7aLblZ0+RTVnq/XMsac04B+I+TyJrBO4pnKd/m7VTyO2Wq1ful5fuWCzirOLAwtRR7t3p2VK5D1M1vqIlu7+blUd2aamzkYVT5muWys+w1qJSCl0KQKYkJIx1mhadrHWVt6mx448qrebBmdyv6n85pi+6BN9etJJF96PVv1k2CLU0e7dKlNDR33EMWKJvIfJnNm9u79bj5mP6sP2D1SQVaCctCn68h8+N+ZTVACxIHABmJCSNdZodv4crVu0Xo+Zg6HL7/frpvKbHa+5OBYznEe7d6ssDR31SR1kH+oaid73ub5zqjqyTSfbP1RBVoE2GJ/TN57bzDI9SDkELgCIYk7+XK1btF6PV2/XO77D6h3o0a0Vn1G6NzX/E2qfU8tq4XLrGlL8k9J29HZou7lVpzsbVJBVoDuMjSrKnsoyPUhJHr/fP9ZlsPit2ZKRekpK8kX9pSbqzrmT7R/qsSPb1NV3TnML5unTFZ9VTkZOxGOs5XZWz71h2HI7yRJL/SU6G3+84nkPfOd8eqJ6u850ndG07Glab9yZ9Ok5xhqfvdRWUpIf0xMbjDgEAIdm5Z2nDYs/p+MtR/WPr/+9/teL31ZDZ0PEYzYYG7W0eJkaOuoDA8iTPaDcqUQfJIjXBmNjTNNV1Jyt1m/eeVhnus5oRk6pNpz/+QkXtjD5ELgAIAYzcmao8VyjjrUe1Wv1e7XlnV/rHd/bYfdfU7lOpbllOuw7FAg6yQw+2w5vcxzeYg0+ybKmcl3ENSEtfr9fr3z4kh6v3q7u/m4Z0xZr4/lfUF5G3iiVFHBPag5AAIAo3Ow++/z5X5TX49WSaUvUO9CrJ2t3qL7jlK6dc33IJxgjTeSZqL99+W91sOGgGjrqo97naExaGq+uvi49fez3qjlbLY/Ho1Wzr9XlZR9L6Xm2ALu4A5dhGF5J/yJpuaRuSZtN06yxbf+qpK9L6pP0I9M0n0ywrADgWLhpCZIRxKzg4vf7dfD0fu2q26nX61/T6c4G/cn8T6koe2rI/cP9Ho7bY66Sff54z3ey/UM9ffT3OtN1RtnpU3RLxae1oLAi4fIA40kiXYqfkZRtmubHJX1P0j9YGwzDKJP0TUlXSvqkpL81DCMrkYICQCzCdZ8lszvP4/FoRenF2rD4c8rNyFNd6/v6j8MP6fX61xxPkhppPFekslrHXTPvGq2ee4O+tfKumMvv5PyxjDMLPl+0c/T09+j5uuf0X+/+Wme6zqgkrqa5UQAADqdJREFUZ4a+sORLhC1MSIl0KV4l6Q+SZJrmq4ZhXGLbdpmkV0zT7JbUbRhGjaQLJe1L4HoA4Fi4ViQ3phSYnT9Hm5b+qZ6v26l3fe/oXw/+XDXN1frqsq/r8xd8OeKxkSYIjVRW67jMzPTAk3+xtjA5OX+ocjk9X6RzvN96XP997Gk1dzfL6/Gqb6BXfzj2lKZlTR233Z5AIhIJXAWSWmy/9xuGkW6aZl+IbW2Soj5iUlKSn0BxMNaov9Q1mepuc8kmbb5iU9LPW6J8fWXWF2Q2mfrM1s/ovTPv6V8P/V9dXH6RrphzRdg5u7526WZlZqZr0/JNI+ohUllDHff4s9sCIczJPdrPv+3wNj385sPatHyT1i9dH7FcTs4Xroznes9p59Gd2n9qv5QuVRTN06eNT2vz7zbrhRO7lJ2V6Ur9jFeT6bM32SUSuFol2f9SvENhK9S2fEnN0U7IfCSpi/lkUhd1l1zTNEvfXH6X/u2tf9G8gnn6/eFn9Mfq3bpi1lW6YPqyEYPqryu9SdfdeJOkkf8NjNRiZR1nr7+15evV09OnteXrY67TB/c9pF11O9XT0zd47gjlcsp+jg/rfdrf8Lpeq39VXX1dSvOm6YpZV+myso8prSstobKnKj57qS3WsJxI4HpF0q2SthmG8TFJh2zbXpP0N4ZhZEvKknS+pMMJXAsAUsZtizfotsUb9H7rcT33/rPynWvSz974qcyz72nj4s/rGxd909HTd7F2642HpXiC9Q306eDp/Xr11B519nZIkuYWzNMn5n1S06dMD+w3np+gBJIhkcD1hKQbDMPYLckj6cuGYdwlqcY0zd8ZhvFPkl7S4MD8H5qm2ZV4cQFgdCXyJN+8gvn68tLNetf3jnbUPKHqZlO/evs/lJORp6vOu1oVRQtHBC/79RIJQbGWO9mBp2+gT283HdLuk6+oradV0uDEsVedd7XmFcxnugdMOnEHLtM0ByT9WdDL79m2/0LSL+I9PwCMB/EMHrfzery6YPpSfeeS7+qhww9qfkG5Tnc26PHq7SrKKtKFM1Zo2fQLlZuRO+J6TiYLdavc8TrbdUZvNh7Uoaa3dK63U5JUkjNDq867JmTABCYLJj4FgAiS1dX22UW367OLbg90sb3RsE97T+3RLw89qAtLluvWis9o+YwVumPRncOuF62latvhbXpw30Mjto/mAs/9A/2qbj6iN08f0PutxwOvz8gp1eUzP67F084naGHSY/FqJAWDP1MXdTc2BvwD+uxvP6U9p3arvHCB1lWulyRNzZ6qyqmGKooW6ry82frcU7dHXPj5S8/eoWdqnnFtcexwevt7Vdd2XLXNNao+W62O3nZJUoY3Q4uLl+iikhUqy52ZlKA1Votuu43PXmqLdfFqWrgAYJQEB4evLP2qcjJy9ekFn9Hcwvl6q/FNne06q9dOvarXTr2q7PRsVRYtUltPmz67MHTQ2LR8k3p6+kalJau1u0W1zTU62lKr91uPq2+gL7Btek6JLipZoSXFS5Wdnp3UkDRW3aNAMhG4AGCUBAeH4IHqH591pT5oOzEUamrkO+dTXma+rp59rd5ve1+/PPSgSnPKVJpbqtKcMs3IKdX6pet1XelNSS9re2+7Tnc26HRHgxo669XQUa/m7uGz+8zMm6WKwoWqKFqoGTmlw1qzkhmSRrN7FHALgQsARkm04OD1eDW3YJ7mFszTdVqts11nVNtco9rmGn3QfkK+c03ynWvSO76PZtmZM32mMvtylZeZr9yMXOVl5Ck3I0+5GbnKzchVmjddXnmV5k2T3+/XgH9AAxpQb3+POno71NHbofbeto9+7mlT07mmwJOFdplpmZpfUK6KooUqL6pQXkZe3PcaC6aMwETAGC4kBWMRUtdEqruJOtZHGpxmoelcoxo66gdbnDob1Nh5WllT0tTR0Z3062WlZWlGTqlKc0s1I6dMpTllKp5SPGLiVqcmct3EayJ99iYjxnABmLQm8lifdG+6ynJnqix3ZuC1/oF+KbdLR09+qPaeNnX0dai9p10dve2BFqt+f/9gq5a/Xx555PF45fV4leFND7SE2VvF8jLzNDV7mqZmTUvqk4UTuW4AJwhcACaMyTbWJ82bppK8MqVNzR3rokQ12eoGCEaXIpKCpvHURd2lNuovdVF3qS3WLsX4OuMBABgDO6qrdOeT67SjevTmHAOSgS5FAEDKYCwYUhWBCwCQMhgLhlRF4AKASSaVp2hgTi6kKgIXAEwydMsBo4/ABQCTDN1ywOgjcAHAJEO3HDD6mBYCAMYA0xsAkwstXAAwBhhHBUwuBC4AGAOMowImFwIXAIwBxlEBkwtjuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AKAcWpHdZXufHKddlRXjXVRACQofawLAAAIbau5RbvqdkqS1lSuG+PSAEgEgQsAxqkNxsZh/wJIXQQuABin1lSuo2ULmCAYwwUAAOAyAhcAAIDLCFwAMMHxtCMw9hjDBQATHE87AmOPwAUAExxPOwJjj8AFABMcTzsCY48xXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDL4ho0bxhGoaTfSCqQlCnpLtM09wTt80+SrpTUNvTSGtM0WxIoKwAAQEqK9ynFuyTtMk3zZ4ZhGJIekbQyaJ+Vkj5pmmZTIgUEAIy9HdVV2mpu0QZjI088AnGIN3D9o6Ru2zm67BsNw/BKqpT0oGEYpZJ+aZrmv0c7aUlJfpzFwXhA/aUu6i61jUb9Pf7sNu2q26nMzHRtvmKT69ebLPjsTR5RA5dhGH8q6dtBL3/ZNM19hmGUabBr8a+DtudK+rmk+yWlSXrBMIzXTdN8K9K1GhvbIm3GOFZSkk/9pSjqLrWNVv2tLV+vnp4+rS1fz99LkvDZS22xhuWogcs0zV9K+mXw64ZhLJO0VdJ3TNN8MWhzp6QHTNPsHNr3eUnLJUUMXACA8YnJU4HExDtofomk7ZLuME3zzRC7LJK01TCMlRp8EvIqSb+Ku5QA8P+3d3chdl1lGIDfiUVDIfEqF1r0psJ3GUitVQgaqRK0SP3BG0HQVqQlN1VREITgjVWxhaBoQQJFmguxJaD401z4118DtoIV+WqqUAUFLdiKgm11vDhn7HSYnsycZOWco88DA2fv2ZvzwZrNfmettfcCWGHzzuG6NcneJCcmc+bzdHdfX1UfT3Kuu79dVaeSPJzkuSTf6O5fXZSKAQBWzNr6+vqia9iwbix7dZmLsLq03WrTfqtL2622Awf2re3meC8+BQAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAYTOACABhM4AIAGEzgAgAY7LJ5TqqqtSR/SPKb6a6HuvvTW445nuS6JM8nuaW7z15IoQAAq2quwJXkyiSPdPe7tvtlVR1K8pYk1yR5TZJ7klw953cBAKy0eYcUr0pyRVX9qKq+V1W15feHk5zp7vXufjLJZVV14IIqBQBYUeft4aqqG5N8bMvuY0lu7e5vVdXhJHflxT1Y+5M8tWn7b0lemeTPM75q7cCBfTsqmuWk/VaXtltt2m91abv/H+cNXN19MsnJzfuq6vJM5malu++vqiuqaq2716eHPJNk81/RviR/vTglAwCslnmHFI8nuSVJqupgkic3ha0keSDJ0araU1WvTbKnu/9yYaUCAKymeSfNfz7JXVW18RTih5Kkqr6Y5O7uPltV9yV5KJNQd+wi1AoAsJLW1tfXz38UAABz8+JTAIDBBC4AgMEELgCAweadNH9RVdV7kry/uz8w3X5jkhOZTMg/092fXWR9zLaTpZ5YPlW1J8lXkxxM8s8kH+nuc4utip2qqkeTPD3d/F13f3iR9bAzVXVNki9095Gqel2SO5OsJ3ksybHu/vci62O2Le13KMl38sK972vd/c2XOnfhgauqTiQ5muQXm3bfkeR9SX6b5LtVdai7H1lEfezIzKWeWFrvTrK3u980/SfntiTXL7gmdqCq9iZJdx9ZcCnsQlV9KskHk/x9uuv2JJ/p7h9X1R2ZXH+nF1Ufs23TfoeS3N7dt+3k/GUYUnwwyc0bG1W1P8kruvuJ6bu97k1y7aKKY0fOt9QTy+lwkh8kSXc/nOT1iy2HXTiY5PKqOlNVP5wGZpbfE0neu2n7qiQ/mX7+fpK3XfKK2I3t2u+6qvppVZ2sqpnLBlyywFVVN1bVY1t+rp52v21+N8X+TN5Uv2FjWSCWwHbtmORPmSz19NYkn8tkqSeW3/68MCSVJP+qqoX3erMj/0jypUxGB25KckrbLb/uvifJc5t2bV6hxb1uyW3TfmeTfLK735zJiNzxWedfsgt0uyWCXoJlgZbYnEs9sZy2Xmt7uvv5RRXDrjye5Nz0Gnu8qp5K8qokv19sWezS5vla7nWr53R3b7TZ6SRfnnXwMgwpvkh3P5Pk2aq6cjoZ+2iS+xZcFrOdb6knltMDSd6Z/PdBlV8uthx24YZM5tylql6dSW/lHxdaEfN4tKqOTD+/I+51q+beqnrD9PO1SX4+6+Bl7YK+KcmpJC/L5CnFny24Hmbbdqknlt7pJG+vqgeTrCXxlNvqOJnkzqq6P5MpGTfonVxJn0jy9ap6eZJfJ7l7wfWwOzcn+UpVPZvJ1JqPzjrY0j4AAIMt3ZAiAMD/GoELAGAwgQsAYDCBCwBgMIELAGAwgQsAYDCBCwBgsP8Alq2wgPdoUhEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num2, mu2, var2 = nums[1], true_Mu[1], true_Var[1]\n",
    "X2 = np.random.multivariate_normal(mu2, var[1], num2)\n",
    "plot_cluster(X2, [5.5, 2.5], [2,2], 'g')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第三个例子\n",
    "$$\n",
    "u=(1 \\quad 7), \\Sigma=\\left(\\begin{array}{ll}\n",
    "6 & 0 \\\\\n",
    "0 & 2\n",
    "\\end{array}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAHUCAYAAADvF/aYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9e3Qc9ZXv+61Hd0vWw7Il2ZbAT9kuv+QHBGxsMwSIQ8jLiY2NYe4kTIYwM5mcIYdk5eScwAqclXNmJpPDOUzu3MlKmLlkTi4QOEpiQni/wTYGjGVbxi7bsmT8kpBly3q3uh73j+pqVVdXdVV1V3V1S/uzlpa6q6t+v12/X3XXrr33b29GVVUQBEEQBEEQwcGGLQBBEARBEMREhxQugiAIgiCIgCGFiyAIgiAIImBI4SIIgiAIgggYUrgIgiAIgiAChhQugiAIgiCIgOHd7igIwloA/yCK4qcFQbgKwB8AHE9+/C+iKP7GsG85gF8DmAFgAMDXRVHs8U9sgiAIgiCI0sGVwiUIwvcB/BmAoeSmqwA8LIri/7A55K8BHBJF8UFBEHYAuB/AvfkKSxAEQRAEUYq4dSm2A9hieH81gC8IgvCWIAj/KghClWn/jQBeSL5+HsBn8hOTIAiCIAiidHFl4RJFsUUQhHmGTe8BeFQUxX2CIPwQwI8AfM/weTWAy8nXAwCmOvWhqqrKMIwroQmCIAiCIELGk9LiOobLxO9EUezTXwP4menzfgC61asKQB8cYBgGPT0DOYpDhE19fRXNX4lCc1fa0PyVLjR3pU19vdm5l51cVym+KAjCtcnXNwPYZ/p8F4DPJ1/fCuDtHPshCIIgCIIoeXK1cP01gP9bEIQxAF0A7gEAQRBeAvBFAP8C4FeCILwDYAzAnT7IShAEQRAEUZIwqqqGLYOOSqbV0oVM46ULzV1pQ/NXutDclTb19VWeYrgo8SlBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAwpXARBEARBEAFDChdBEARBEETAkMJFEARBEAQRMKRwEQRBEARBBAzvdkdBENYC+AdRFD8tCMJqAD8DIAOIA/iaKIrdpv33A7icfNshiuKf+yQzQRAEQRBESeFK4RIE4fsA/gzAUHLTIwD+gyiKrYIg/CWA/wTgPsP+ZQAgiuKnfZWWIAiCIAiiBHFr4WoHsAXA/06+3yGK4nlDG6Om/VcBmCIIwkvJz/+LKIrvOnVSX1/lUhyiGKH5K11o7kobmr/SheZu8uBK4RJFsUUQhHmG9+cBQBCE9QC+DeBPTIcMA/gpgEcBLALwvCAIgiiKUrZ+enoG3EtOFBX19VU0fyUKzV1pQ/NXutDclTZelWXXMVxmBEG4HcAPAXxBFMUe08fHAJwQRVEFcEwQhF4ADQBO59ofQRAEQRBEqZLTKkVBEP4vaJatT4uieNJil28A+B/JfRsBVAM4b7EfQRAEQRDEhMezhUsQBA7APwH4GMBvBUEAgDdFUfyRIAj/DuB+AP8K4DFBEN4BoAL4hpM7kSAIgiAIYqLiWuESRbETwLrk2+k2+3zN8PbO3MUiCIIgCIKYOFDiU4IgCIIgiIAhhYsgCIIgCCJgSOEiCIIgCIIIGFK4CIIgCIIgAoYULoIgCIIgiIAhhYsgCIIgCCJgSOEiCIIgCIIIGFK4CIIgCIIgAoYULoIgCIIgiIAhhYsgCIIgCCJgSOEiCIIgCIIIGFK4CIIgCIIgAoYULoKYJMR2tqD6jq2I7WwJWxSCIIhJBx+2AARBFIbYk48j9urLAID45q0hS0MQBDG5IIWLICYJ8R13pv0nCIIgCgcpXAQxSYhv3kqWLYIgiJCgGC6CIHKG4sIIgiDcQRYugiByhuLCCIIg3EEKF0EQOUNxYQRBEO4ghYsgiJyhuDCCIAh3UAwXQRAEQRBEwJDCRRAEQRAEETCkcBEEQRAEQQQMKVwEMUGhlA0EQRDFAwXNE8QEhVI2EARBFA9k4SKIIiAIa1R8x52I37yJUjYQBEEUAWThIogiIAhrFKVsIAiCKB5I4SKIIoASiBIEQUxsSOEiCI/EdrYg9uTjmsuOrFEEQRCEC0jhIgiPUDA6QRAE4RUKmicIj1Aw+sSG0mkQBBEEZOEiCI+Q+29iQxZMgiCCgCxcBDGBmSjWmkKeB1kwCYIIArJwEcQEZqJYawp5HmTBJAgiCEjhIogJzERJN1EK5xHE6lWCICYOpHARgUE3oPDxaq0p1jkrBavTRLEmEgQRDBTDRQSGfgOKPfl42KJMCvyIc6I5yx2K/SIIIhtk4SICoxTcQBMJPywsNGe5UwpWOIIgwoMULiIw6AZUWPxQlmjOCIIggoEULoKYIJCyRBAEUbxQDBdBEARBEETAkMJFEAZKNVFoqcpNEAQxWSCFiyAMFNMqPS9KVDHJTRAEQWRCMVwEYaCYVul5WXVYTHITBEEQmZCFiyAMxDdvRf8TLUURfO4lr1MYcgflxjS3S+5SgiAmAmThIogipdhXHQaVWd3cLmVwJwhiIuBa4RIEYS2AfxBF8dOCICwE8BgAFUAbgL8RRVEx7FsO4NcAZgAYAPB1URR7/BScIIhwCcqNaW63FN2lxVoiiSCI8HClcAmC8H0AfwZgKLnpYQD3i6L4hiAIPwewGcDvDIf8NYBDoig+KAjCDgD3A7jXP7EJgggERQESCUCSwMgSIEmAJI+/TiTAKAqgqpCWroD04H8DAHBHjwCqqv0B4/8BgGHG/+t/pvcqxwE8D/A8xq67HmPXf1p7PzyM+K1fQvxLXwXY0omAIKscQRBm3Fq42gFsAfC/k++vBvBm8vXzAD6LdIVrI4CfGD5/ID8xCYLIQFGAeBzM6AiYeBwYHQUzOgomPgqM6ttHtX2SShRkOfVaU6JkoIzDlMtD2ueK4txvWLAswPNQOT6pnHFQ+Yi2jeeBpNKmRqJAWQxqrAxqrAwoL0u+jgFlZVDLkttjscCUuFK0yhEEESyuFC5RFFsEQZhn2MSIoqg/wg4AmGo6pBrA5SyfW1JfX+VmN6JIofnLg0QCGBwEBga0/4ODwMgIMDqq/Y2MAG+9pf1dfTWwZAkQj/vT9zBQEWGASESzOEUiKWsTDh0C3nsP2LgRWLdO28Zx44qKlcXK+Nps9TK/1v9kOWVZs/xLJAxWMxlQZGAM2l+uMIymdJWVaX/l5eOv9ffl5UBFBVBVpf1VVmpj4MTddwF334VYHuJ5gb57pQvN3eQh16B542NwFYA+0+f9ye12n1vS0zOQozhE2NTXV9H8WTE2BmZwAMzQEJjBQevXQ4NgRkcdm4q98CL4jpOQhkcQn9GoueJ0q41uwSkvt7bmRKNpbjvNSqS9r5s1DT19o9pnJotP9R1bEfvoI8QbrkD/j3/q+/Ckxzpty76zLKcUMEt3pySBkSRtzOOjFlY/w+t4XBvzwVGMPxu6Qy0rh1pZCbWiAmplVfrrigrtfWWVprgWAPrulS40d6WNV2U5V4VrvyAInxZF8Q0AtwJ43fT5LgCfB/Be8vO3c+yHIFIUXSCyLIO53Aemrw9s8v+4QjUAdnAQGHNpguG49Jt2RQXU8imaslQWg1pWDmX6dESfexbx27YjvmW7Zp3RLUr5UFEBDFu7EoN2jXmKdeI47S8WgyFCLO21J3SXbFI5i/7hd4jt/D3GNn0WibUbtO2jIwYFeTCpHI+AGR0BLmRfB6TGYkllLPlXVQW1pgbK1BqoNTVQq6dq50MQxKQgV4XruwB+KQhCFMARAP8HAARBeAnAFwH8C4BfCYLwDjSjPwUyEHkTSiDyyEhKmWL7LmlKVd8lbVt/f3pwuBU8P37TraxMvVYq0i0kKC93VJ7kxQJG7/4rH0/OmaBTU8hNTZBbP4Tc1BRYH7awrGYRLC+HCiD66iuI7t0D7sQxKHPnpZ13bGcLos/9AfHb70B8060p5YsdMihig6bX8bhmZevtte1fra5OKmDTtP/TpqWUMpSVFWYcCIIoCK4VLlEUOwGsS74+BuAGi30+m3w5BsDBP0AQ3gjE2qKqYPovg7l0adxKdbkP7KVLmvUqm6uPYaBOnZq6Yao1NVB0F1PSUoWyMn+sUCFiZ1n0w+LItbeD6+0F196elyx+EN9xJ/jWD8H19mp9GM/VqOx/5TZNWcZM2C4xUFVgZCRl7WSGhsAO9I9fX32XwAwMgOnrA9fXB5zqzGyirFy7pqZNg5q0iqWUsqrqkr+uCGKyQYlPiZIhb2vL4CDYCz3Jvwup11ndftFoygXEHTmMyK63Ef/KFsS33g51as2EdQkZFRs7y6LVdq8KkVcluvyRhxFpOwS2u8t/hUs/h6T8+cgJhgGmTIE6ZQrU+noAgGzeR5LAXL4M9rLBctpncFGPjoDpGgHbdT6jeTUWg1pXD6WuHlg0Fyxfob2eMsXTORMEUThI4SImHvG4QbHqAdOjKVjM8JDl7mpFJZTp0zOsCMrUGu0GlrQkVD/1BKLvvwe1eipGv/ktX0X2qqgEHc+WpkzZKBtW2726fYstm76dPLnKmXWeeB5qbS3k2trMA1UVGBpKubFZY6xgby+YkWEwZ8+APXsGOPERyoa0FatqRSWUujoo9fVQ62dAqauHUlsHRKOeZScIwl9I4SJKjtRNbPsOjG24AWzPJ+kK1mXrVWcpq0C9ZhnQ/9xaBYIMIPeqqAQdz2Y8Vy9KSNBB9iP33gfFwgJVrOQ8TwwDVFZCqawErpydaR0bGkpd9xWJISjtH2vX/tAguKFBcCYXpVpTo13vuhJWVw+1trakkskSRKnDqE5Bv4VDpeWxpUvgy5tVFcxAP9hz51D5t3+FaOt+SAuaEN9iESrI81Cm16ZuLmp9nfa/iONewrRwFcPS9CAsdsWwqtWNDPnKmZq/ZDwi22O07PaAvdirpdQwE4lAntUApaERSuMVUBobtXQWRMEohu8ekTv19VWebiikcBG+4PsPx9gY2K7zYM+dA3f+LNhz58AMDQIAePEIuLZDkFc0I7FuvaZYGZ/cp02jJ3cP5Dt3fig21XdsRezVlxG/eRP6n2jJWZag2wyCXOQ0jnn13Xdlnz9Z1haFGC3BPZ+A6ctMj6hWVUNpbITccAWUhgYosxoKlk9sMkIKV2njVeEilyIRPqoK5sIFTbE6fx7subNaMLvpYUAtK4fS0IDEhushNzRCaWjU0in4QDFYQ0oVP9ybQbgiS6W8Ti5yGsccd9+VfWeOg1pXB7muLt01OTQErusc2HPnwJ7X/piBfnBiPzjxqLYPy2oPMw0NmhLWeAXU6dOL1lJMEMUMWbgIX/D0pCbLYM+dBXeqE+zZM2C7zmv5ioywLJQZM7Wn7VmNmrtjWnA/9KViDQkCL3NnpZgGpazGdrag/JGHAWixW1ZtT1ZF2ZOFyy2qCqa3N/ngk1TEej6xePApgzKrAcqVsyHPnadZwSboat2gIQtXaUMWLqL4UFUwPT3gTnWAO9UJ7vTHWm084y7V1ZAbr9DiSRoaocycVVBXRrFaQ4pNobCyZgW10jD25OOItB0CAC1Q3krhCiMZbhEQyJgzDNS6Okh1dUDzKm3b2BjY7q501/7gALjODnCdHYi88xbUWAzK7DmQ586DPHe+FoxPFjCCyIAULiIQmP7LmgWrsxOx5/8Aft8HkFc0QxKWAgCUunrI8+ZBuXJOUQTrFkN6AkvrUZEpFIVUTOM77gTb3ZXWn3mMilVRDoqCK+DRKJTZc6DMngMpuYkZ6Ad79iy4Mx+DPdUJtrcX3Inj4E4cBwColVVJ5WselHnzQv9uE0SxQAoX4Q+jo+COieBOdWg/whcvpj7i930AvuMk1KoqxO/7T1DmzqUfYQssrUdFplAUUjG16ss8RsWgKBeSYlDA1apqyEuqIS/RHp6Y/stgT53SrNenOsEMDoA/fAj84aR1srYOyrx5kOfNh3zlHK0GKEFMQkjhInJDVcF2ndeebE91Av29iA2Ol8FRYzEoc+ZqT7mNjYg++wzid/wp5OUrwpO5yDEqV0ZLRqnGlAVhjQlKAS02160dxaaAA4BaPRVy80rIzSvHF8B0ntQUsDOnwfZeANt7Afy+D7TYzMYrNAvYwkVQZswk9yMxaaCgecI9sgz241PgTxwDd/w4mMHx+aqonoL+qXVQdFdCQyOlZsiDQgbxBxW4W6hzKHRaimJTzoo68NqwQIY71Qn2/DlAGa9AqVZXQ160GNIiAcqVsyfdb0ZRzx3hCAXNE/4yNgau46TmLuxoTyvmrFZXQ164CPKCJlSsWY745XiWhgg7rG7gugVDbmpC9R1bPd3ci0UhyOccnHBT69FLW2x3FxIrml1ZjorBreeGorgOOC4VA5bY+CdAPA7u9MfgTp4Ad+IEmP5+LeRg3wdQy8q135NFiyHPm0/5v4gJBylcRCZjY+BOtmsJRk+2p60oVOrqtR/ExUK6OyAaBUAKVy7YrfwDgIoffA9cb2/atlzay0s+D+kZ5KYmcO3taSWBdOuRX/IA7mo9emkr0nYI8Zs3BVJwOyyCUAzzVuJiMU2pWrgI2KSCPX8O3PFj4I6LYC9eBN92EHzbQSAahdS0CPKSpZDnLwB4ulURpQ9dxYRGIgHuZDs48Qj49hPpStYVV0JaJEBetEjLhUX4it0NPPbk4+B6eyHX1nq6uctNTZBbP4Tc1OSLfMb0DOwPvqfJahPMrrz5OlhJStvHjYLidCM3fy43NUHeuyeVoDMfl6Xt+NvIVCqB+kEohr4qcQyTLCl0BRI33AjmwgUtXOGYCLbrPPgjh8EfOQw1FoOsK1/z5pPyRZQsdOVOZlTtCZM/0ApePAKMjaU+UhqvgCQsgSwsgVo9NUQhSw+vVgDb4tCmAtJu4drbwfX2gmtvdy90NvmS6RnYzg5wvb3auVkUreZbP7RUELMpKPpYsd1dKaXOTb4trr0d3OAgMDhoKY+n87ORT++Tb/3QVq5iJgjFMEjrnlpXh0RdHRLr1oPpuwTu6FHw4hGw3V3gP2oD/1GbpnwtXQZp1RotVx9BlBCkcE1GRka0H7CDB7RM0kmUhkZIwlLIggB1ak2IApY2flkBcr1h+n1T1OUwKpJW+wDw7G7Sxyqxollz6RnaTlNcTedklaPLb4xKZL5K3UShUNY9tWYapHXXQVp3HZhLF8GLR8EdPQL2k27wrfvBt+6HMqsB0qrVkJYso1QTRElAqxQnC6oK9uyZcWtW0u2jTqmAtKIZ0spVUKfX5tw8rbYZpyiClT2Q69z5cZ7Z2rBaOVjI0kJBt+0Xk+m7x/T0gD/UCr6tDczoiLYxGoW0ZBmkVau1MkMllGZiMs3dRMTrKkVSuCY6w8NaEsKDB8D2XkhtlufOg7RqDeRFi32pgzbZfjgKeSP20pfdvtnayHXu7BQipwB7t1jJbNVnKad08IPJ9t0DoMWcHhPBH2zVSoUlUWbMhLRyFaRlK4CyshAFdMeknLsJBKWFIDRr1senwB88AP7YUUCWtc0VlZCaV0JqXknB73lSyNQAXvrS92W7u9IUCz9SJ9ilrjCW3TGuqrSrf+gWK/eVlbvUiwu1VFI6EA5EIpCXr4C8fAWY3l7wB1vBtx0C+0k3oq+8hOibr0MSlmpWr8YrSsrqRUxcSOGaSCgKuI8OI7J3N9jkTQ8MA3lBE6SVqyE3LfTFmkUUvqag2770fdjurvQSOA5tOK4SdChabVS25MpKKPPmBzI2lkqYh7iiQsybWyvaRLS2hYFaW4vEjTcjcf0N4I4f06xepzpTKSaUGTORWLce8mJh0iVWJYoLUrgmArIMvu0gInv3gOnrA6DVO0tZsygA3neKLTWAuRRQbGeLZmHSFYxs8j71lGO+L6fUCWx3V2qF4tDf/7SgY+MlrqsQ8+bWikbWNp/hechLl0FeukwLtD94APyhg2A/6Ubsmd9Bqa1FYt0GyEuXkeJFhAIpXKWMJIE/2IrIe++C6e8HACjTpyOxdj3kZcvJmjVByHZjtkur4EmxeOwx23xfTjUdrVYZFlp5sBof4zb9faFkc2tFK5UEqqWIOm06EjfciMSG61MPo2xvL2J/fAbq7reRWHsdpOXN9BtJFBTuwQcfDFsGnQeHh8ec9yKAsTHwH36A2DO/By8eAROPQ6mrx9hNm5DY9Dmos2YV/AmuoiIGmr9MYjtbUPHQA0CEh7xkWW77R3hgeBjxHXdmtFHx0ANawtEZMyE1r7Lcx6n9ippKxC9dxsh3vpuhkOjt87vfgTJ7dmbbSdlG774Hwz980NU5+o7F+PCt+8B2diCxYSMib7+lKV/Dw0CE9zQfuSAvWYb4bbentW817lb75QJ997LAslr6iDVXQ62pAdt7AcylS+BOHAd/+JBWeqh+RmgWL5q70qaiIvaQl/3JwlVKxOOI7N8H/v33wIwMA9BW5STWb9RWG07CwNCg42C8ZkDP+Nyj28gpVsqMXXJU29WKVvJs347+G2+1bd+YiwpAxirEfMfdvLJRlzPbnJrPz7yfMfmrcYzMli+/VlQ6Qe7DkOE4SM2rIC1vBnf0CCJ7doHtvYDoyy8ismc3EtdcC2n1VVS/kQgUsnCVApKEyHvvIvbM78G1nwAjJaA0NGLsls8hcePNUOvqQle2wnpS0y0wGB5G/LbbC96+Y/9ZrFOW2OxvZymTlywDIrymDBmtVga5dKsO37oP/NEjUGbMxOjd94zvWxGD/MQTtu0rs2enZIo9+Tiie/eA++QTxzF3a92reOiBtDb5tkOOc+pl3OObt6YsSWbLl94v23ES0ReeA9+6D+W//Hl+FkkrvF4HHiAriQcYBmr9DEhrroJSPwPspYtgL10E19kB/uABrdzQrIaCWbxo7kobsnBNMNiT7Yi9+hKYS5cAAPLsOUisWw9l3vzQlaxiIOg4GKf2HT83ZGmvvmOroyXOqcyMvo/TZ1ZWHTlpqbIq0pytjI1RJv79vZD37oFaM83ynI2WJ7dWHbus8dnm1O24mzFbvvR+AaSNkZPMqWNM55gtWF/f323bRIAwDGRhCeTFArj2E5rF6/w5RF9/FfyhgxjbdAuU2XPClpKYYJDCVaQwA/2IvvZKqjivUluHsZs3aYoWkSLoVWdO7bupE+hF+bDtJ4uCYZmbykIuuakpzcWW4qmnwIlHobCsYxkbvYZhfO11zoqh2+BxU3oJY/kgN0pqqu/ksebzTHM9mlywxn4Vi2OdyMhF5lExJkKGYSAvXAS5aSG4kycQfe0VsBd6UPbEryEtb8bYp28CKirClpKYIJDCVWzIMvh9HyC6+22tmHQkgrH110P61DW0oiZkvMaL5aJ8ZOvPLuWBrVXMZoVhhrXtscfAnzkNAJBrayE3NWUoOkZlJp7lPOwUGtfnbF5d6EF5MVvyrNqwkylXxd18nFfFmCgSGAZy0yKMzJ2PyN49iOzdA/7wIXDtx5G4/gZIq9ZQKgkib0jhKiLY0x8j+vKLYC/0AADkxQLGbvoM1OqpoclEyRnHsbNQ2LqRXCgf2cbXtr888zxlbL/rLiROnwGgBY9nU2bigHV6CFOh6VxdZ2alhO3u0pK47mxxVGz011ZWqkIpOVkXOBRZ7jbCAp5HYsP1kJYtR/SVl8B1nET05RfH3YwNjWFLSJQwpHAVA0NDiL75Ovi2gwAAtaYGY5/5LOQFC0MWbGK5QczKjdeM4ICasgClfW4zRm5usNnG184iYrXdTekdHbmpCXLrh+PnsX07+gyrFPn396Z/bjgGUC2tX8bEqYC9ZSrrWFjMD3v+HLje3owyQdnGVrpmLYZ+/JO0fb1gVw8yq3JMDyYZlPKYqNOmI37b7eCOiZqbses8yn79K0ir12Bs4w1AeXnYIhIlCClcIcO1H0f0j89qle95Hom11yFx7bqiWZ48kdwgGQHOHi1Fcm1tKuDaiNeyOcb3GcqPsV0P7i+roHc7N6QxcNwKq8/1bWjdD663N61WY+zJxy0TpzophBnjZDE/dglZ3RyfK7EnH08lkTUqehSf5Y2SH5NkYP3I/AWI7H4HkQ/eA7//Q3DHjiH+pc1Q5swNW0KixKC0EGGhqojsfgfRF58HI0mQ581HfMs2yMKSoorVcpuc0c/lzV6ThbrGvDTf7VL95H6JDRuhJFfnyUuWpeSUVzRnTfppTmFgfM9euoTI4Tat3WR6A/P5uxqPCA9+9zuaUmRIlaAfG331ZUT37kmdL9txEuzZMyj7zePg6moxPH+R/ThZjAEzNJhqT25qAtvZgfjWbRj51r2pa4YXj6TkTt18s6WSsJmfke98NzWOTmOQS+oF8/jyrfvAike19Bnf/lvHBLSxnS2IJpPPGtNtFAq7715g3yO3BJgKo6BwHJR58yEtEsBd6AHbewH8R4eh8pG8C2NTWojShtJClAIjI4j98RlwJ9sBhkHi+huQWHsdpXlIEtSTsZOLz6n+XmxnS5rVx0vag2z/za9zscQZrTBWbZlL78SefByxtkPAmdPAY48BN95q60ozjoFxrBSThSuye1eau9HLogFLl6ihz+o7troaAy+rGS3l3LzVdiWmbRxe0iJmlW4jTMK2ME20mDW1vh6jO/4UkbffRGTvHkTfeBVs1zmM3fJ5IBYLWzyiBCCFq8Aw3d0o29kCpq8PavkUxBqVJH0AACAASURBVL/4ZSjzF4QtVlFRKDemV8Wm/JGHEWk7lMrdxHZ3aYqMh7QHdu/T9jedfza3Y7Z+zG2Zg87184jcdRcAe1eaU196zBfT16cpcfrnHhYN6LUgja5KK3mtAui9Yp5nJ4XYyR2a7Xr1I44p1zYmUjhA0cCySNxwI5SGRkSffxb80SNge3oQ/8pWqLW1YUtHFDmMqqphy6Cj9vQMhC1DoHBthxB76XlAkqDMakB881ehTq0JWyxfqK+vQrHPn10slb6qzbi6zerGVnPTBkTaDkG6cjaYkeFUElGrVXt+olt3gupLnzvdwsX09QHQFm9YBY2bx0mXL7GiGcrMWZ4UA/OxuuJlda5+jYNXBSaffr0cayeXUxul8N2biDC9vYj9vgVs7wWosRjGbv0i5MWCpzZo7kqb+voqT24piuEqBLKM6KsvIfr2m4CiaEWGv7IFmDIlbMl8oxRiEcyxVHqsUfkvf64VgK6Zhv4nWmzjTdSaGmB4GIwsgT/VCbm2FiPf+W7w8SnJWBi5qSmQsjP63MlLlmH063+B6BuvIXpgf0b5Hn382M4ORA63pcoG6fFLI/fe572Atan4tT7GlnE/LsbB7pyN2wGAbzsEeUVzXuWWvJyf07H6Sk99XNPi3BzasPru5RO/5eXY0OPEwmTKFEgrmsH0XQLX3QX+6BHtYXrOXNfhIaXwu0nYQzFcxYYkIfbbp8F1dgAch7HPfFZLokcUHC9pFiyPN8RyKXm6ibyg9+smjslILjE8sZ0tYLu7IF05G2pNjW2eq8juXeDEo6j4j/9Bi3eyKhdkExNWcf/3EWt5GvGt2zD045+4TvfgZhzc5B4DvKWtyDUWycpila2ouN2KzFz6zyd+y8uxYceJhU40irEvfQXKrEZE33odkb17wF66iPiXv0qJUokMSOEKEllGbOdvwXV2QJ1SgfjWbZQ4L0Q8pVnI4nbyur9feI3JySm7vSEA3Oy+yghiT8Zr2aVtsE2v0PK0VkKo5em0fFmOsmXJdu+UCd9poULF/d9H7PFfQ62ZhuEf/Vdf5tAqVYdtzjabWLtcySd+y8uxFCcGgGEgXbsWyqxZiP2+Rcvd9cc/YOwLXyKli0iDFK6gUBTEnt0Jrv0E1PIpGL39Tqj19WFLRbjE65O7MaA+MIXLo6XD6/66dSuxotmyvE9a24age7vkoHJTk2Wh6/jWbSkLl5M8VqsJ5b17MmqKOmXCt8tdpn8Wa3ka3OAgMDiYtZakF+I77gSfLDOkt2lrZfV5RV8+7Xk5dqKtRMwHZc5cxG+7HbGnnwR/5DDA8xj73Odp9TmRghSuIFAURJ97Fpx4FGpZGeLbd5CyVWIUbKVkEWXj1i1Scm0tIrt3paxTbq2CehtG151VeoWhH//ElWXLajWhrsBwbYegJBUmo2XLTlF0SgUR37otZeHKZc7t0lro8sV33FlUc00Eg9J4BeJbtqHs//wG/KEDAM9h7DO3kNJFAKCgef9RVURfekEr0xONIr5tx6RwIxZj8sV8+rZL+GrXph7sbUx8adzXmATUeJw5kN9vrOQ1b0vNnSF5qjJjpra4w2uguDHAe0VzfokvTcHi8pJlUGbPBttxMpVkVFec9AUP+gKIjOSv5oB0U9uJmzZh5N7vYvQv/yYnWe3m0XgdBTXXxRx4PRmD6tWpUyE3NIIXj4I9dxZIJGxT/xTz3BHOUNB8yPD73gd/sBWIRDC6dbuWiXgSE2ZQbRB9e6mb6CZQ2xiIns2F5wZzMLqVvOa6h1b5wNxaYZySlZrb92rhscwrZheHZ7BIst1d4MSjqLlpQ6oYtzkg3e84PDcWUb+tpqkan/fcDRhqYRYTkzWoXpk3H/HNX0Xs979F5P29UOrqITevDFssImTIwuUj7JnTiD37DKCqiH/5q1AWZE9UOZEwPqmlPdXma+XIhyBKi3hp04W1x5yawmyZqfrWN1H2q3+DWlPjaDkr/9n/0uocdnZAmT1bKzvU1JRWjqjioQcQOdyWltLCOHduSzkB6VYdRHhUPPQA+NZ9aSkbjLKWP/Iwonv3gNv/IaJvvOZLygKzvPKSZYi+8Fx6Wosdd6ZKBGW76ZstUF6sM27GzcvYuiEl79AQhr+aPR4OCMnaFOHBdpwEMzSYdg1PBtTptVCrqsCdOA6+8ySkBQuBysq0fcjCVdqQhSssBgcRe+b3gKIgcc1azwnwJhLGp9r+J/LLCp6zDAHFy7gJEjb2bQzgzpq53cL6YVzlh0ceTrNAWVrODMHodkHk5n6q79ias4XE2FYqoD0ZY6Wfr7498ubrUKqqAQBM36WM1XvZMKaXAJA1tkyXRw/oz5blPtv5AD4Www7qWtTlTFYKcJQjBGuT8Rpwql4AFFdMox9IzavAnj0L/mAryna2YORr3wDKysIWiwgJUrh8IvbqS2AGByDPnoPEDTeGLU6oFMNS8VJzZVq6z3bcCX7vHm31nGm7XuZGmTkTcm0t5KamtGD02M6W1L52/aTyWUV5VwqX+WZoVACNQeuR3bvGS/DsuBORN14DK0lQh4cQv3kT5Kam8dQQbm7CBsXTWBPSjWxeyHCHuryOnZSEoGuD1tdXAS6ylYf1vfTS70R0QY7dvCn1fY2+8Zq2cpGYlOSscAmCcBeAu5JvywCsBjBLFMW+5Of/BGADAP2XYLMoipdzlrSI4dqPgxOPaknwKPdKwZaKZ82V5fHm4mfNO7tcUF77NcdUGbenLErnz2mr9gxFtfV9nJJturGQGI+1uhlaWdJ0RY79wfcw9Pc/hdLQCPbMaaj1M1L7SNeszTgvO+xSUKTJGcCN2u117NR3MTyAAOGlcPCUZqJIxspXIhHEv7gZ5Y89Cv5gK6QVzVCunB22VEQI5KxwiaL4GIDHAEAQhH8G8G+6spXkKgC3iKJ4IR8Bi55EAtFXXgIAjG24Hmr11JAFmjxku9F5zkHlww3bKReUU7/6+2xB6DrmrO/Zijo7Bfpns5CkHZsleWha0L8p/9Twj/6rpdLoxb3U99qurPsGeaPOp3g1UBy5qoznALhfGFFoimGsgkCtrUXi2nWI7NmF6EsvYPTr3wA4LmyxiAKTt0tREIRPAVguiuLfGLaxABYB+IUgCDMB/Ksoiv+Wb1/FSGTf+2AuX9aW0V99TdjiFC1BxGb4cZPNxSrllzzGRKN2FiQnpGvWgmtvz4iRSRvvPLOO6+4QIFOR1G+QxnI7+j75uviyJZMNUoFwytllxjYnmU2x7zDIp7QR4Q+JdevBHzkM9kIP+EMHIK2+KmyRiEKjqmpef4sXL/7t4sWLbzRtq1q8ePH9ixcvnpJ8/cHixYtXOrRVeiQSqvqP/6iqP/qRqp44EbY0xc2tt6oqoP0vJrLJ9ZvfaNt/85vC9O22v9/8RlWrqrRjV6/W3q9ePf7aqm3z8V7OS29r9Wrt9b33Zh5vbNOq/VzGcvXq8X6N7axePX7+t96a37XlZuzuvVdV6+q0/17Q26mryy5fUNeZ3Zzkej4TlaC/50ba2rT7xSOPqKosB98fETSe9KW8LFyCINQAWCKK4uumj4YBPCKK4nByv9cArAJwMFt7PS4CP4sJ/mArot1aksjRqnpXgasTlfr6qoz5S7NCbNmO2JiE+JbtiBfROMVMclnFLMXHJPQHkOfI3DduvHU8eN1ijPQ8W2r5FPAD2ucJSUbfjbei+heParL+4lH033irp/Oymru0fmfPRay2FuqFXvCtrZDfew9cb2/6uBhk161dxs9T8jmMpXFFYmL9BnC19ZplKClf9S8eRay1FQAgV1ZiaMt27TgP11bGWCTbM4+dPHsuuM98Fmx3FyIXLiDe9hH67VyvZqvYzhaUnz4DrGjWzqO9PdWe2dLlNDZO1mG7+Utr94mW8fn5xaOIOZxPvpTSakO316Yv1F2J8kg5mNPnEd+9D9M3XlNy9z1inPr6Kk/75+tS/BMAr1hsXwzgSUEQrgLAAtgI4Fd59lV08O+/BwBIXLuuZEo3FPKH0I/0EEHLa3YHOcUsBdm3E/rKPrkyjsSKZgBaELlRRmNiT2C8XqDdecV2tgC/fQqxLdttZeHa28H19iKxohFxYUmai8zyvLLEejmu+DMWvJ45a9w9aXDPycmVm1pyyfQyOm4wj4XuLgVUTF86H/Gt29D/REtKcZQrK7PWlzQnkwWQeh+/eVNq5ajR7eplNWSu8YV27bqZi3y/d6W02rCggfosi8Q1axF9+UVE9r0PbKQwlMlEvgqXAOBk6o0g3AfghCiKzwiC8P8BeBdAAsC/i6J4OM++igqmtxds7wWoZeWQhSVhi+MaNz+Efik5vsRYFfiH2xgEXsgndDdjbsyzZa5FaJk53UbJMqeGwKsvIzYmpY6xW82Yz1g4KZdGhSqRVICMCo5xQcLQ//yZ5QpHt9eteY71lY/Tl87XAv0f/3Vy1acKhefBDQ5CmjkrFSunn4/eH9vdlZbJ3iqzvbHfbKk6ssnr9Xtku+DCzYKFPL93pbTasNCB+tLyZkRffxXsmdOAKeULMbHJS+ESRfEfTe8fNrz+CQDnCrUlCnfiOABAXtBUUqtNXD3d+qTk+PFD5ucPtyulxiIIPFt6BcccTC6P0cecE4+i4gffSylVxn3tij5bubPMCwGyrXaMRXnEt2z3VrbIps9cg/+NCpW+IjFtDmwURqs22O6u7KsKTXOcSsCaVGjV8imaZau2FqwkpfKcRXbvSi1wMPZnzA2m9zduNbMeLy+EsXIv3+9d0DKXkssyg2gU8tx54NpPAKIIzFkctkREgaDSPjkSfet1MP39SKzfCLWuLmxxXOOqvEgOJXGCKlHhZzkUT8WDLcbA6vhsbVoWTbY7Jtkfd/QjcAMDYMWjGLn3u65kziixk+xTL+hsHDur0jgV93wD8Q9bEX31ZW217apVKP/lzzPK9GQby7T3O+5M/efFI5blidyMt14Whj17BvyB/Ri9+540pTWjTI2+f2cHIuJR53k2FOvG8DAG/59HMfLt70CdNQsYHkZiw0YoNdMw8p3vIvL2W4ju3QOpeRXkFc1pZZNG774Hwz98MLO80N491uPjsWyQE0F89/wuQ+Q3QRd9DxpGSoA7cRzRCIfhpslblaTUodI+hUBVU0+w8py5IQvjPxM1F47rOCJTKRl9mzGFg5s27VxLclMT5NYPITeN19rUx3zaVcuBM6eh1kxzLXNaTJZNn0aZrKxPeuxU/OZNKdeZuUyPXZ/m//q5mGOb7Eq72CZm1VMytB0CzpxOT3thcR76/pG2Q2nn75RCwkuOMLN70y7fmjlOznztlFKMUzFSSi5LK1L3jfPnwxWEKCikcOUAM9APJBJQp1QA5eVhizOp8eJacKtIpgVuJ4POjUHQTu621GdpAdnIiPkxZ4cHkJEk1I3MVvvYujBNNyo9aN5cmidhXF1nESxu7tMuhkwL8q/UAtytFECTUmZuQw+SV5MFuI3nZ/xvtd2snLHdXWCTmfnNMuuLC8wuWf38rfJoZVWCTXFyujKbGr8d4/nNKu7/ftZ+ipmwXHul/lCoTq0BeB4YGADicSAWC1skogCQwpUDjP7EXlsbsiREEJYC803ezmqU7WajfwZoxZZ1xc0c82MmlxuJUx1Bu9WiurKD3l5wN29KrczTlQPz6rpsxaatxsJNsL1xbK0Uu8juXeAGB5EwrEjMNk52yifb3QW2swPc4GDmPJquoVTZpORqSNlQz1JvP9/VgsZ+dCUwm0WxWCFLXY4wDJRp04Hhy2Av9kJpaAxbIqIAkMKVA+ygljeFyviETxCuBa69HdzgIOJrr8uwaJjdcHaB9Wx3FyJthyyVKydrgFergV15IP79vYi1PA1p9ZpU0WhdoQHGrXaoq7N1EeqvjaV6vASsO1rnTO7QoIo8G12NQ3//U9tVmMb/nHgUGBzUFO+11/kaQG50MxotidnSbRQTflZomKyoU6cCw5fBDAwADWFLQxQCUrhyQZa1/zwNX9gE4VrIuAFnWeFn/A9kX7mmt2Uk24pFq/2d5DUeqytJaN2Pi0c60lb9AUhZlrh//mfEsyR81GUof+Rh25qNugxsd5d1jFiW1Zp6egarm3di/Qaw588hsX6D4zhkI5u1zc49mqu7zE3ZIWPMnNXKU3M7xWY9chPHRmRH1e8f+v2EmPCQxpALepJTVQ1XDsI3nNxyVti5r/T/XpUr/T2gplxs5uP1YH49d5SVHHpskLR6DdC6H/Gt2wCMu0o58SjGPv+FlHJTvX17KrN9ttQQ+mdWwe9GJUWxypFl0a5eK1F59WWwABLdXamUEOZ4t1jL05CuWeta8bCbT2OslhO5KvNu6ha6scwaa0nq7eaqfPmtvJV60HoxwOj3jxJJmk3kDylcOaDyEe1FwttS7GJ+Yg2TYhgXtzmcMo6zuLHbWXYsV9ZZWKfk2lrLoHpzML9dDFPK+jBzFi4e6Uh9prtKMTgIub3d0jJhFVRvDri3y7iu9+/WGqjDmt4bA+kTK5pT42HMmp9hMXJRcNpvl6Xt6kobt6wRL8oc29mRUr70Yz3L6vO5l3rQelEwlrx/kKdk0kAznQPqVC12i710ydNxFGBqTTGMi5NLzA6jolb+yMNg+vqg1tSkLFBGK4W5DI/eh9XKPKtYHuOqRzvLgl36CrfHm+Uxyt/32i7EN1snhXXC6jxH7r0PMI0ZkB5Ib9yWLc7LvN1K6bFKx5EPbhPF5npNj9x7H9ik4qkAtgst3FAqFqliePgqFGyfdv9QptaELAlRKEjhygFlurY6kb3Yq7kVXZqES+VHr9DkOi5+/jhbucRcZaY3KGq6BcKcMwoAmL4+11nus53nyL33jVt6bFyU5hQExn2NCozeJ556CtW/eDQt5khX+tjz51Lym885m9KntxHZrbkIjS5Qp/O1cstauUyNsWTjsqTXQzSi14S0SsfhBvN456LAeU1jAuTnSjS2VWgFJpfvZzE8fBUESdK+UxUxqNOmhS0NUSBI4cqFKVOglpWDGR0B039Zy6nigmIzwxfL06QfsTJ+yW+UxY0lx6io6dYaACmFYOTe+6AkY5HcBpNnO08AWeODLAP5TbmudOWQ7e7SjnvssYw2lTdf18raVFYCANSa8WvcSTHUg/XV114FoyoAxvOZubnesl0Pae7CRx7OcOea6yHapanIBb1fPT1GLgpctpWtXly0+VKI734u38/J8lDK9CYf1qdNI5fiJIJmOkeU2bPBHT8G7mQ7pDVXhy1OTpTy02Q215lfePnxt1LUlCcfT+W98hJM7kYOo2XNaO2xukGb84iZs+jjrrsQH5NSbeoKk1xbi/jWbbbJT82kguqTFl9GVdKSnrq93syJR+1ipKyUWHM9RKNs8R13Zli9PFmczOkxXFr6nJS+ML6HXvvMRUHLRXkqtofSoOA7kkr6nDnhCkIUFFK4ckReuEhTuE4cL1mFy9enSYNbyq8fzKyJRZOus8SKZk8rz5zaNZLrj7/uagLUNCXFSzC5ObUAAPDv781QQLKtHMyWKymxojnlXsT27eg3pYUwj48x+amVHMbzVkdGwQ4PQYmVYeh//izTauNwvZkTj+qWOHMAf/kjD0OaNh383j2ouUlLHxHfug0Xj3RYLlYwJm4150vTt2dD/1xPjwFkT4lguUjC4joIw6rjtc+crFWTRHnKBe7Ece2FQHUUJxOkcOWItGAhogwD7uNTwMhISZb48fUH0eCW8k3hyvIjn3OQu0MpGT/QXU1o3W/Zj5sUFFZuRN3Nl96eCoXnAWSmKLHKlVRz04bxhKwuZdKtiXJlZSpFg9V56eedWNGM+MxZmZYpg5JoPtaIPrd64lG2swMV938/rV8gmcGf58FKEti2Q2AAxFqextCPf5JxDubErVb50szjoMtqfq3MnGWr5BpxE+Nlp/w7WfmM++Ceu4EsedSs8PrdnyyuvkLADPRrsZE8DzQ1AZfjYYtEFAhSuHKlogLynLngTnWCP9AKad11YUsULia3FJB/nEi2G5ZVkLsTRmXLqrhzRmoBrxnfTRYlu9WGubgR09x8RqtX636wkgS+dX9GG16CurOlxTBaE6WZs7KuotT/u4lHcxMXp8+XrmxlzpsKvnU/lIbGlIXLqj29b/O4Zig6LuLl3I6rU4yXnfKfdp1mKfeTkjXKe1a4vDLRrVWFjGfl930AqCqkhYuAaBQAKVyTBVK48iBx7TpwpzoR2fc+pE9dM7mDH23cUvlYvdwEJXu5ERhjmczlXczyAvBsCXObfduNtcAuFkvPyp7aJxmzZKVo6ONnTBqqB/HbKUuWcVEuFCk7ma3One3uAiceRc1NG9JWL9oldjWveDT2VXH/98G37kdi/QbbjO1Wshnf69az+NZtWePl9Nf6deQULO+kmJlj6yy3J2PorK6VlOXtrruyykE4U7A4ungc/AHt4Ui6Zm1w/RBFCffggw+GLYPOg8PD3hKJho1aUwPuxHGwfZegVlVBmVWaBbFiO1tQ8dADQISHvGRZTm1UVMSQMX8RHhgeRnzHnbm1m+/xNu2NfOe71j+qER5sx0kwQ4OI7N0D/lSntkovEoFaU+Msg0t55SXLEL/tdsf2jPMCAHzbIfBHjyC6d4/Wz223I3HTJox8+ztI3LTJUh5+9zua0pjcnxePgG87BHlFc6r/iooY+mc3IX7b7dpqxOQ58OIRVDz0AOQVzRj+4YMZ8jpdN/rnfOs+lP/y50CER3zzVkRfeA7RA/vBffJJSi4AqHjoAUT37knbro9V4qZN2nGGcweAqr+5B1xvL9jODox8+ztZx9MOYxuJz3w2NT7xzVtTMqTNmcM8p8776BHwpzqh1ExLyWueH8vr0bB95Fv32l4rukwV116V+d2zkCef7/eEx+/fGhv4D94Hf/IE5DlzIV23wfp3kygZKipiD3nZfxKbZHyAYZBYex1if/g9Im+/BUlYWpKxXEE93eXrhvDbjeGmPfb8ufE4pJs3pYKqneJ1/HZJmN1NACxjjrJh5Upzmmur1ZZ2+5rbMo+BnjRVD35Ptb/DOgGrvp3p60utvDTKb2l9ymLhs8Jy5eDWbYg9/muo5VNcZXR3uo7s4sPctpPPde9Hbc7JSCFcpsxAP6LvalbaxNpJHoIySSGFK0/kJUshH9gP7uNTiL71BsZuCTaWIggoIFbDnOHcS4yY72VjbNxNXhW6DFeal1QXDvsaP8+2GEGtmYb42uvG9zcFz5tl1RU99gffg9LQmFKA9BQbRoZ+/JMMV6LT6lbzPA39+Cfg2tu1QPieHsiVlXllo3frgg0Cy5WRAXy/iyWHXykRfe0VYGwM8qLFUOYvCFscIgTIpZgvDAOl4QpEDraCPX8O8tx5UKunhi2VJ9y6uLLhp2k8DBdIbGcLoq++DGXGTIzd8jlE3n4r5QJzNTYBuz/9mCNgfK51dyEiPGJXrYb8xBMZY+7Up/HzioceQORwm6asfue7kJcsS7knR7/9t2kuSV05ixxuA9txEtEXntPcn7oLs6kJbGeHVtJmxkxIzatS4+rm2qh46AEtDcTud6DMnp12nNzUpLn4zPOku18HBsCOjdm7AT2MsR/Xgd35GrfHrlo9/t2zuA79lEdHH2Oje5ewh2s/rv2mRKOaNTZWBsDf302i8JBLMQTU2lokrl2HyJ5diP1hJ0a+9udARUXYYvlKIZ9ow0oEmcrrZZP2IBthuD/zIW2BwN135TzmVrm+UpYVi/QS5Y88DO74cbDxUSgcp5U8SlqwdPdtorsLQ3//U+t0CS5XeNqlgbBb0KC3pQfth2ntdSp4DmTOn06hVhOarWZk8bKH6b+M6HN/BACMrb++5B7ICf8ghcsnEus3gjv9MdgzpxF7difi23YALBu2WL5RSCUozESQbHeXbdoII3Yr6oIk1zQVVvunuQPN7z3043Zlpr5vqt4kAFaWIdfUIFFTA048Cranx/Gc3K7w5N/fi1jL0ynXYK4rQ8PAmKg1vnVbRtJaIHP+bNsKSBHKUKYpTswaSUJs5+/AjAxDnr8A0jXXhi0RESLkUvQLloU8bz74jw6DvdADyDKUefPDlso/HFxmfprGg3CBuO1Td4PZrmRMYlxRZ3SLZZM5X1eplavMzf5Wbp80d6BhlaLuHnRyFzm66KxIrgJVozEo1VMhz5mLxPoNiLy3F3x3FxhZglxbi+H7fzR+Ax8eBiJ8atzcunjLf/lzRA63pVyD+urC2JOPexr/oNzbWds1ri6Nx6FWVqatKgUy58/uu1cw11+BVvmVFKqK6KsvgTtxHGp1NUa37Ujm3RqHXIqlDbkUQ0Stqkb8S5tR9tQTiOzdA7V8CqRrJ0aulWJ5+g8at+dpXGkHwNXTvVWJGU+yWbjKnPY3/vfSj/m4jKSw+kq87i4oM2e5a9dibGtu2qBZFGMxKIsWI7F+Q8pFmUjWiXSzctDVOeRghQnKcpO1ioJhUYHXSgpmCmUtniy/D16IvPOWlpCY4xDfvAWYMiVskYiQIQuXz6g1NVCqp4I/cQxcZwfU8nIoDY2B9xt2rp0wntQKbX0wbo9v3orRr/8FRr/+F2m5q4wB2sb8U/KSZeBb94E70ApuaCgni4O8ZBmU2bNdWxKsAuTdWCetLIxGSwkifGqBAYCM3FheKPvVv4H75BPIS5eh77VdKP/lz7XVgjXToFZWIrp3T0bgvBssraQurTBp87+i2VfLjVvLoNnimq3/bN+9MKzFBBDZswuRXW8DLIv4l79q6+0gC1dpQxauIkBuXokxWUL0pRcQfeUlIBKB1Lwq0D4nSwyFm4DivPtwEahsl7XcuJ+5LAvX3g5WkhzjwwD72BuvlgS/akemxQwlY7H0HFNuSytZYc58b5fp3Y/5dTt2Rkvk0N//1DE2zQteYt4AshyVIvz7exF5+02AYRD//JcgL6YC1YQGWbgCQpnVADUWA9dxElz7CSjTpkOtnxFchyHHUBTqSS0tJmXHncGcs91YJrfLHHli+wAAIABJREFUTU1pliu74xMbNqasGLx4JGUVGr7/R443UbNFKVdLnlW6hox9XMxdWob6pNVHbmpC5O23xt2MOVgczRYY43vj69jOFlR965so+9W/Zc3674vV0yJDv2/4+D1NWVLLYhiev8gnAYl84PfvQzT5UDZ26xe070oWyMJV2pCFq4iQPnUtGElC5K03EHvuDxgbHoJ09TUAw6T28WsV0WR5EjYnlXRcRZfD+DplAHfKwG7O8K4fk7IKuYwR0//rMUxsd1dOcV/6/1yuD73OoFo+BfyZ0wDGE5Cax8GNxTHX6z1theMjD9u374PV0zx/uWCsz2hMzJqLhdIxiWsBilcTDqiq5kZ85y0AwNimWwL3ahClBylcAZNYtx5QFETeeQvR114Be/4cxm75fGq1ymRxBfqF5xtWAOPrJhDZ3K/X4GXjeerpJ3KSNU9FXM9JJlfGM8rU2LkBvYyLW+I77gRvKBGUas+kkPgRJO7HQ5A+brGWp9MULs+pPbIF1+s5sKh4dbiMjiL2x2fAtZ8AGAZjN94Mac3VYUtFFCGkcBWAxPqNUGrrEHv+WfBHPgLb04P4V7ZAnV7ryw2CsMfNijvPbbpQYjKUELfxQwbZAO2Gm1i/AcrMWaFcI8ZahXaKA//+XlT84HuIb93mGJeU88pJg9VJbmpC9R1bLeP48lEw9XPSE7Aa+3U6xngtxXa2QC2foimpphqPXhXObOOln2t9fRXQM+B8goTvMJ98grKdLWAuXYJaVo74F78MZUHuZaGIiQ2jqmrYMuioPRP8R4O5cAGxnS1ge3uhxmIY+8KXIS+cGLEX9fVVKJX5011h8Zs3+RoQ7QdG2QD4IqeTgpnL3Bnl1FNVyLW1uHikI2c5s2G1WEK3uPmV2FM/p8SK5pSC69Sm1bWU7foKIhFpkN89qwcAyiavwR1uQ+yl54FEQrteNn8Vas00T22U0u8mkUl9fRXjvNc4FDRfSKZMgbS8GWzfJbDd3eCPfAQoCpTZc9LiukqRQgR/+pYGopiTNBpl85iSwG58nJJf5jR3BjnVqVPBdnYgvnUb2IF+31J1pKXX+Nn/QuRwW8ZiCbeJUJ36MKaAGL37Hgz/8EHHdBoAUslcmaHB8WB+w9iY2wikrmEe3z2n75Tx2uHbDlH9RACQZURffwXRN18HFAXSipWIf2VLTuXcKGi+tKGg+WInFkP8y18F/95eRN96HZE9u8CeP4f4FzdTYjwH/IrHKtYFBlbWDz/i1bK5pWI7W4DfPoXYlu1pLjHbQG3DZ7r1Jr55a8rd6LSgwAup9BrJ2C25stL1YgmvfQDjiwHMnxkT1WbEjBkWCygWyWiLPUbTST67NB1hE1btRmZwALFnfg/2zGmA4zB28yZIq9aU/AMzURjIwhUGDAPlyishXzkbXMdJsD2fgG87BHVKhZY6ogS/vAV5UgvYMuXVguZ34tWqb30T0b17wHacxOjX/8J7/zbjk82qUvHQA+BfejGtXFA2i5jVZxX3fx9Vf3MPIrveAtvdDTUaAyNLWdM32J2PnvqBOy6CP3okmVyVAdvfD3mxgKG/+6mn9txYqGyvKYv0EJZjY2qjIKlLDOeXV1oIh++UXZoOv8g2R9k+K1jJIh1FAX9gP2K//y3Yixe1qiK33Q55kZDX7zVZuEobsnCVEMrceRj92l2IPvcsuI9PIfbcHyAfOoCxz9wCtb4+bPECx+tTatCWKa/WiLCtF34EjMd33InYgf3gLlxIlQvKGqid3JYWuJ5ckce8+UYqsSt/5rSlxcfpfPRgdfb8OXC9vVqc1r33ZU3RYHcdOc2Pm+tPaWiE0tCYPTGrbuna2YLqO7ZCbmpKFZwO8pr1Iy1E2NbebHPkZoVmIaxtbNd5RF96AWzXeQCAvKAJ8Vu/mJMLkZjckIUrbGJlkJc3Q6mZBu7sWbC9FxA52ArE41CuuBLguLAldEUuT2oFf0p1wqsFzWeLm17GZfTue2xLChm38637wHZ2ILFhIxI3bcqpT3nJMlQsW4z4pcup87ArCWRUUCJvv5Wau8SGjWA7OyCtuw7ygqZUwlfHBLFmkvFQyoyZGLvlc6mksQDAtx0CMzJs2Z7tdeRUcN3h+qv61jcRaTsEZcbMlGXNyVoYe/VlsJ0djsXPfSF5fvxffKN0E586WBjtPitIyaKREUTfeBXRl14AMzgAtaoaY7d+AYnrb8goQp0rZOEqbbxauGiVYjExMoLoO29qBU9VVfuC3/QZrTREkbsZc1ltE1YcRqHI9/zMaQrMq95qbtqASNshJFY0o++1XTnLaTd35tV2xvduVgeaj891PPR25NralNXLOA56u3JTE7j2dvc5rhzk8Tq+xjJKua4szWWMaKWbz6gquMNtiL7xGpjhIYBlkfjUtUis3+iboqVDc1faeF2lSC7FYqK8HGObPgepeRWiL78I9vw5xHb+FvL8BRi7eRPU6bWBdR2G8hO4izBkhS5fl6N+fGJF87iSY1qmHyTZEpsa82IB7gKucx0PuakJcuuHkFavgQQm49z1trzWjHS6/sx1Ht20B+SXoT5sN/Vkh+npQfSVF8Gd/hgAIM+eM2lCPIjgIYWrCFFmNWD0T78G/mArIm+9Aa7jJMr/30eRuGYtEtesBcrLfe9zIv7Qh31O+caZ6IpGYv0Gy1WAXhUCrzgpJI4r3EzHj8upYvrS+RnJVO3g2tvB9fZCAmNrNYo9+XgqF5hf42GU3+3KzXwfIigRckgMDSGydw8iH34AKArUKRUY+/RNkJevKHrvAlE6kMJVrLAspNVXQVokIPrm6+DbDiLy7m5EPvwAiTVXI/Gpa30N2pyIP/Rhn5Pdzdet5U1XNLj29vE2TVamfLOqy01NwOlTaWkhbI/xqVzR9KXzLcve2B7noh8r65vteeRg+cw1uNsrYQexTzaYgX7w7+9F5EArkEgADANpzVUY23hDIA+2xOSGYrhKBPbsGUR2vQ2uM5nJOxJBYtUaSNeuhVpZFa5woFgEL9hlIjcrAn64RO2KKOvxSXIsBi4et4xTCkKebDIVilwqDXixcBUa+u55h7nch8h774I/eACQZQCAvHAREhuuhzJzVsHkoLkrbbzGcJHCVWKw588hsmcXuBPHtQ08D6l5JRJrr4NaPTU0uUrthyO2syVVFHrk3vsKeqO0u0EHURJGtyaZy+7oCpcSKwMbH81QuGI7W1DxH/8DuMHBvIPyrQhKSXHTbtgKkltZ3MpZat+9MGEuXUTk3T3gDx8CFEWzaAlLkFi7HurMmQWXh+autKGg+QmO0tCI+JZtYLq7EX13FzjxKPj9H4I/eADS8mYk1q6DOm162GIWPcacT17zReWLndsoa0b4HN1WxuLTRvT4L7mpCVNOn8LIlu0Z/XGDg6778YrxfPT3uboDreosZmvXq9suSAXNb1dlMSmTxQRz4QIi7+4Gf+QwoKqaorVsBRLr1kOtqwtbPGKSQApXiaLOnIn45i1genq0H5KjH4E/2Aq+7SCkpcuRuHYdrazJQnzHnWC7u1KvC4HTzTCbIuBUnseu3aEf/8TSbWfsa0p9FeKmp2zj+Izce5+Ls/OG8Xy8KBZW+6ZtMyRm9bpq0UuffuEmyayX6zPshSLFBtvdBX7vHvDiUU3RYllIK1Yise46ejAlCg4pXCWOWl+PsS9tRmLDRkT2vgv+8KHUn3LlbCRWroYsLAEikbBFLSrCCE72nMne5eo3P26yVn0FOT5W7btRLLLV9jPKXn3HVstVi7lYgNwqPjm1bcpSbzzW7SrJXGSd0IyNaQ+gB1rBnj+nbeM4SCtXaQ+iU2vClY+YtJDCNUFQp9dqWZDXb9CCQQ+3gT1zGrEzp6G+9jLkZcuRWLkG6owZYYvqO2G6Ubz07fVm6FaR8uMmG6ZlxI1ikbY9aRUzHput9EtaOzmcp1vlM58xNB+bsWDB7bUwWVc5qqpmzTrQqrkNx7Ts7WpZGaQVKyFdcy3UquqQhSQmO1TaZ6JRVga5aSESV30Kak0NmKEhsH2XwJ4/j0jrh+A6TgIAlGnTfS0bFGaJCrclgvwuNm3Xt10/nsuRuCwd5KbdrIWAK2IYHpMt+7I6zs042u3jdKxxPBHhU/umFI7hYfBthxzn23ZMgiyAnk/b2Qpg33Z71rYndXmY0VHwhw4g+tILiOx+R3ODyzLk2XOQ2PgnGPvcF6AsXATEYmFLasmknrsJQEGLVwuCsB/A5eTbDlEU/9zw2TcB/CUACcCPRVF8Np++CI9Eo5BWroa0cjWY7m5EDrWC++gw2HNnET13FpHXX4W8dBmkVWsKugw6CFy7fLzECrksF2PVt5uiyeYVklaWHT+tFV6TlGY7zs042gWvO8phF9uVxZXohSAtQPm0bZckNvV/slqurFBVsOfOgj94APzRj7T8WQDU8imQlq+AtHI1BcITRUnOCpcgCGUAIIripy0+mwXgbwF8CkAZgHcEQXhZFMV4rv0RuaPOnImxmbcAN9wE7ugRRA4dAHvmNPjW/eBb90OZOQvSqtWQhKUlmezP7c3Ii+tNv9nLrR9mDbx2cmfZtW1eIRm0Sy+rTE89hepfPGqpVDrFTBkx5tfyoji5kTlDISkS5cNcaskv1zYpWBYMDY3HZl3oSW2W58yFtGoN5EWLAZ6iZIjiJec8XIIgrAXw7wBOQVPc/osoiu8mP/sygM+LovhXyfe/A/DfRVF8P0uTRZMQbFLwySfAhx8CBw4AIyPaNpYFmpqA5csBQShJ5cs3nnoKeOwxYPFi4Ngx4K67gO3bnY5y3/bf/Z32+j//Z61dvT8/+3HL5z8PPP88cOutwHPP5d5OfT1w4QJQVwf0jN8QPZ2bWZYwx8UNRnkBf8aRGGdoCDh6FDh8GOjo0FYaAlqVjdWrgauuAmqDqzFLEA4ULA/XMICfAngUwCIAzwuCIIiiKAGoxrirEQAGADhm5aQEcAWEKQeu3gCsWgvumAi+7SC4j08BrW3aH8dBnjcfkrAU8sJFQFlZ1uaKOYFfTkH1N96q/Rnx6/zMbfcMpG+z6Sdfa4rdONTfdRfiYxLiW7ZnpIfw0i5WrQbfuh/xLbdhyNiOi3NLtbVlO2IGWap/8aiWDHZMQr95PooAo7wA0mQPrE/TPBbzdy8nhofBnzgG7ugR7TdJUbTtLKv9JjWv0n6TOA5Q4N/3MgQm3NxNMurrvVV5yUfhOgbghCiKKoBjgiD0AmgAcBpAPwCjJFUA+vLoiwgKnoe8bDnkZcs1k73hh45rPwGu/YSmfM2ZC3nRYkgLFwOVlWFL7YmJkptIPw++9UMoDY0pt6Tb1AG247B9e17KjN5u/OZNadnsHY9zEbdW7GkOvLo63Sr/WTPQT5Dr2Qgz0A/u+DHt7/TH6UrWgibIwhLtt2cyW92JkicfhesbAJoBfEsQhEZoVq3zyc/eA/DfknFeMQBLAbTlIyhRACoqIK1aA2nVGk35Oi5qytfpj8F1nATXcRLRl1+E0ngFpIWLIS9aBHV68Zvzi/2m7SXHEp+MKVMaGhG/eZOrYH3jAoA4/B+HXMfXjeJQTLFMfqQfcassZduv2K9nV6gqmN5e7Tfm+DGwXefHP2NZyPMXQF6ylJQsYkKRj8L1rwAeEwThHWjxV98A8LeCIJwQRfEZQRD+CcDbAFgAPxRFcTR/cYmCUVEBafVVkFZfpSlfJ09oT5+dHWDPnkH07Bngzdeg1NZBXtAEXLUCKJ8GRKNhS56B3zdtv/N+ecmxpO/vNsDd2H4csC3WnE9NP6NcTufgRt5ixQ/LkttzzpqBvoiUUE/E4+DOfAz2VCe4k+1gL14c/ywSgTx/gfYg17SQlCxiQkLFqwlvjI1p1q7jx8CdPAFmVNOjKypiGBqVIF9xJZS58yDPnQdlVoMWiD/ByFZkOhuuknrapJHIR8FzOr6+vgrxz3zW9pzcnG+uYxIUuY61VRt6ahCnFCFhUbRxQLIM9vw5cKc6wZ3qBHvu7LirEFoaB7lpIeTFAuS58yZlNYyinTvCFVS8mgiWaBSysEQrFyTLYM+cBneqE7jYBRzv0GK/Pj6FyNtvQi0rgzJ7DuS58yDPnQ91+nSA8XR9ZlAMxXn9dqE5WSzytay4sYikpXEwl/lxcb7FZq2yda/mULdRTw2SzUJIQHMTXrgA7lSHpmSd/jiV8R0AwLJQrrgy+XswD8oVV07IBzKCsIMULiJ3OA7K3HlQ5s4D6qsw/PEnWrxX50lwpzrBXLqUCoQFALWqOvVjK8+dl1PwfTEEDOfq0slVKfFTmbFTWI3npFurjNsdFbYicnNV3P998Hv3QLpydsaY5ZIHzGjZItJhBvrBdnamrFjM0GDa50ptHZR52gOXfOVsx9XOBDGRIYWL8I/ycs09sFgAADCX+zRXwqlOcKdOgRnoB992EHzbQQCAUlcPZc4cyA1XQGlshFozzdECVmyWFC8Ug1LiKlDd5zF2m7XfL2ItT4MbHIQci2XGuXmYg3zmy40lthistZ5QVTAXL4I9dxbc+bNgT38MNpkUOLVLReW4BWvePKpfSBAGSOEiLPHjZqBOrYG0cjWwcrX2Y/3JJ8kn4Q5wZ06DvdAD9kIPeOzT9i+fAqWhAUrjFZAbGhHZ9x5iLU9n3KhL4ubkI3ZlcoKysrku1uw2xYGLrP1+Kh/xrdtSGe/Dwmv5o6K8poeHwXWdA3vuHNhzZ8F2nU/FbKaIRiEbwwbq6vIOGyCIiQopXIQlvt8MGAbqzJmQZs6EdO1aQJK0J+VzZ7Uf83PnwAwPgTvZDu5kOyIAYi1Pge84Cfm9d8ENDADxOOL/f3t3HyRZVd9h/OmZnp1hZ2d31t1ed5eAL2hOMFEqKyoWqCgSFIvgS4nGqBXFRC1SlpKXiiaGSoqEhBIUiS+RYJEEkJRSROIblICIqCERVEjhUV6qVFhgFndnd4ed2Znumz9uz2xPz+y89PSZnjv7fKq29s69t3vO7OnZ/vbvnHvuWW844uZ9HPY2OYnmcy1Uc1sOO1y5gKG5+Za0WEwQG7nwYkYuvHj2Ni8h2C3msYWb91at0vXE43TtzANW985HKO3ePeO0bN0Ate3b86r00UdT27Y9X4BU0rwMXJpV8jeDcpnasc+gduwz8q+zjNLe4fzT9M5H6N65k4kXnJAf2riRbPdu2LiRtZddQnXrNmpb80pYbdu2fNhiFX+qnut+je3SShBpfo20elHAbM811a6Frlu1yGrbfM+31Meu6Hlv037XHqV756N0Pf4YTExMP6+nh9rTt1Ldtj3/Xdu+3SFCaQkMXJrVsr8ZlEpkGwapbhikevzzGAdG3/r7+afuRx+ha+fOqU/d3b/4eX4FVF3W10e2uUJt82ZqlS353LDNlVW5lk+qfmkliLRzVfjD/VwLfc4Fr2W2lDaupIrUQo2MTA3dd+3aRdeuIUq7hiiNjc04tbZpE7VtR08N69c2V6xeSW3kOlxqi2VbT+app/IJuzt31ueVPEZp9MCsp2brBvIQtrlCrbKFrFKhtmnzEbneT6PZ7sW391+uKtYE7iap1ypbySqVAYYeeXIqWJV2DdE1lAes5qsGJ2Vr+6lt3ZrPl9y6LR8aXIUfUFY61+EqtsWuw2XgUlt07D+OLKM0sp/S0PRP8V27hmB8fOb5pRLZ4OBUFWyyIpYNDkL5yCj4Ni9S6n/6nV+4dcGBb3yc0u7dDVWrIdaP7mPkl4/Nfv6aNQ2v9cqh6m9/f5ofRIvi716xufCpjiylEtm6gbya9axnH9qfZZT27J42jNI1NETXr57MhyXra4Q1P09t40ayDYN5KNswSLZxI7UNg/mn/1UyT6yVobEiV4AWotPDhVNDolnG2KtfQ9fwnjxYDe+htGcPpeE9dO3eTWn/LG/O/b35mnhP29RQzc0ru9n6DavmdSsVnRUutUVhPqlVq/laQkNPTKsSlPbunXbbkWZZb28exOoBrDGQZQPrCz3XZSF91+4K0GoPcHOqVinVg9RkoFpzy0303P4tqsf/JtXnPPfwj+3qItuwYVp19mm/8SyGqj2Ffg0eqQrz/6ZmZYVLmkt3N1mlQrVSodq4v1qltG/vtKrCZJWhNLyH0tgYpScehycep/ltrfunP6F8//2Mn/IyDr7iVWTr1uV/+if/7idbNzBjyLJdoaMdzzPvDarbvRjqSl+DqlXj4/kQ9/76n8ntkZH8ysDhPXm4b/qgm60f5OBZr8+3+/rIBjdSGxycUW3NBtbPXBalMgC+aUsrnoFLq9aigkh3N9ngxvyNrvlYlsGBA1NBrPcr/8mam77OxI4TqT77OMr3/pjyww/BxHg+Kf9w3+KhB+i+7z4mTj6Fgy9/JX2fupw1P7yb0v79jO940VRIY82axf2cbQgv8z1Hu6+O7PQQ3qKNjU0PT/v3Td8eGcmPNy8MOptSKa9SbRisv+YGqU3+PTl8LWnVMXBp1WpbFaVUgrVrqa1dC9u2U/7bj9Lz4x9Rq2xh/8Ufp7ZlC73X/DvjZ7yG8ZNOnqps9HzrVnruuJ3qC06g+sxnU77nbsoPP0Rp/CC1TZvJjjmGieE9ZFu20PeFq6e+XdbbS9bfD31H5dt9R0FfL1lvX9N2H/T1MXbW2VCtMvaW32v5R1zuANSRNahqtTw4jY3mwWh0NK9cjh6A0fr+sfr++jFGD9C1f//0mzDPpbt7qqLZWN3M1q2j1r+OrF61cvhPOvIYuLRqpQoRzc879sZzGHvjOTPO6/v8FfT85H5qR/8aT13+z1SPPZbe667l4GvPZOKkkxl/+akzKyX79+chYJZ1kgDK8X6677uX6m89n4lw/NT+iRNfTPfPfsZRn7wU6iEt6+2FvnpI6ymTdZehXCYr90C5O9/uLsOWDUw87/mM/93F0NND185H6+c2nFMu58tppFrlP8vyQDQxARMTlKoT9e0qpYnxhv3V6eeMj8/cd3C8KTiN5kFrnurT4f5t84Pl6cPE9e1a//Th49V0cYWk9jJwadVKVUVZ6PNOC2alEmPnvI2xc+YJf1mWB4WREUqjB+rBYay+PUbPN2+i/PBDZGv7qZ7+mjxQjB6oV27GDlVuhocX/gP199I7MnvAm6GrK6/iTAavUgkoHdouNW1n2aH5So3bHNouTQatZbiAZyqENlQIs94+st5eem79JuWHH6JW2cLYR/66Hlr78iDV22uQkrQkBi4pkZYCX6kERx1FdtRRzBY/Rs/7ANnh5qXVavVhsvpw2IED9WrZaL4m2US1oSqUV4+YmID+Hqq/2pdvV6uUxscbqkzVadUkajWo1Vho9JizajTbz97Tc6iiVu7Ot3t6yMrlfBiuXK5vl+n5wV2Uv/NtatuPpuuxnYy/+nc4eOpp+fnTKnx5qKK3d84K3ei+4al/2+pxc1wpKEktcFkItYWXNxfXQvpu6gKEc97K2Jm/m++crXo1WbWqV7AG3v0Oem+/jbFXvJK9V107exWsVMoD1iKHKyeXqqhu2kT3k092bNHSTvN3r7jsu2JzWQhJbTftAoQ3vHne8yc/xo29/Z1QLufDqm1e3XxyyLZ63HF0P/hgca54lHREMnBJmlerFyCkvBqxI1c6SlKLDFyS5mW4kaSlSXSNtySpVb1fvj6fo/blI29OmrRaGbikFvmmeGRajn6fnDPXe921yb6HpOXlkKLUolV7P0DNaTn6vXC3PpI0LwOXkmvXTZpXGt8Ui6Odr8Hl6HfnzEmrj0OKSm65hkeWe4hv7Ow3sfcL16/oN8YiDnumaHM7X4NF6HdJK48VLiW3XJUgh/hmSvFvkrpimaLNViMldZqBS8kt1/CIb6ozpfg3SR1sU7R5ttfgah3qlrQyGbi0ajjvZaYU/yapg+1y9aMVUUnLycAlaVFWS7C1IippORm4JB2R5gqODjdKajcDlyQ1cbhRUru5LIRUEEVc4qGoxt76NsZOO93hRkltY4VLKgirLstntcxTk7RyGLikgnCStyQVl4FLKgirLpJUXM7hkiRJSszAJUmSlJiBS5IkKTEDl6RCcXkMSUXkpHlJheLyGJKKyMAlqVBcHkNSERm4JBWKy2NIKiLncElqK+dYSdJMVrgktZVzrCRpJgOXpLZyjpUkzdRS4Aoh9ACfB54J9AIXxhhvbDh+PnAuMFTf9d4YY1xaUyUVgXOsJGmmVitcbweejDG+I4SwCbgHuLHh+A7gnTHGHyy1gZIkSUXXauD6IvClhq8nmo6/EPhwCGEr8NUY40Utfh9JkqTCK2VZ1vKDQwgD5JWtK2KM1zbsvwD4FLAXuAH4TIzxK/M8XesNkSRJWl6lxZzc8qT5EMIx5GHq001hqwR8IsY4XP/6q8BvA/MFLoaG9rXaHHVYpTJg/xWUfVds9l9x2XfFVqkMLOr8VifNPx24GfjjGOMtTYfXA/eFEI4HRoBXkU+wlyRJOiK1WuH6CLAR+GgI4aP1fVcA/THGz4UQPgLcBowBt8QYv7b0pkqSJBXTkuZwtVlmabW4LI0Xl31XbPZfcdl3xVapDCxqDpe39pEkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJGbgkSZISM3BJkiQlZuCSJElKzMAlSZKUmIFLkiQpMQOXJElSYgYuSZKkxAxckiRJiRm4JEmSEjNwSZIkJWbgkiRJSszAJUmSlJiBS5IkKTEDlyRJUmIGLkmSpMQMXJIkSYkZuCRJkhIzcEmSJCVm4JIkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJGbgkSZISM3BJkiQlZuCSJElKzMAlSZKUmIFLkiQpMQOXJElSYgYuSZKkxAxckiRJiRm4JEmSEjNwSZIkJWbgkiRJSszAJUmSlJiBS5IkKTEDlyRJUmIGLkmSpMQMXJIkSYmVW31gCKEL+DRwAjAGvCfG+EDD8T8E3gtMABfGGL+yxLZKkiQV0lIqXK8H+mKMLwX+Arhk8kAIYSvwAeBk4AzgohBSFozDAAAIjElEQVRC71IaKkmSVFRLCVynAN8AiDF+Hzix4diLgTtjjGMxxmHgAeAFS/hekiRJhdXykCKwHhhu+LoaQijHGCdmObYP2DDfE1YqA0tojjrN/isu+67Y7L/isu+OHEsJXHuBxldKVz1szXZsANgz3xMODe1bQnPUSZXKgP1XUPZdsdl/xWXfFdtiw/JShhTvBM4ECCGcBNzbcOwu4GUhhL4QwgbgeOC+JXwvSZKkwlpKhesG4PQQwneBEvCuEML5wAMxxhtDCJ8E7iAPdX8ZYxxdenMlSZKKp5RlWafbMCmztFpclsaLy74rNvuvuOy7YqtUBkqLOd+FTyVJkhIzcEmSJCVm4JIkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJGbgkSZISM3BJkiQlZuCSJElKzMAlSZKUmIFLkiQpMQOXJElSYgYuSZKkxAxckiRJiRm4JEmSEjNwSZIkJWbgkiRJSszAJUmSlJiBS5IkKTEDlyRJUmIGLkmSpMQMXJIkSYkZuCRJkhIzcEmSJCVm4JIkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJGbgkSZISM3BJkiQlZuCSJElKzMAlSZKUmIFLkiQpMQOXJElSYgYuSZKkxAxckiRJiRm4JEmSEjNwSZIkJWbgkiRJSszAJUmSlJiBS5IkKTEDlyRJUmIGLkmSpMTKrTwohLABuBpYD6wBzo8xfq/pnE8CJwP76rvOjjEOL6GtkiRJhdRS4ALOB26JMX4ihBCALwA7ms7ZAZwRY9y1lAZKkiQVXauB6+PAWMNzjDYeDCF0Ac8FPhdCeDpwZYzx8/M9aaUy0GJztBLYf8Vl3xWb/Vdc9t2RY97AFUI4F/hQ0+53xRj/J4SwlXxo8YNNx/uBy4FLgW7gthDC/8YYfzzX9xoa2jfXYa1glcqA/VdQ9l2x2X/FZd8V22LD8ryBK8Z4JXBl8/4QwvOB64A/jTHe3nT4KeCyGONT9XNvBU4A5gxckiRJq1Grk+afB3wReEuM8UeznPLrwHUhhB3kV0KeAvxry62UJEkqsFbncF0E9AGX5XPmGY4xnh1COB94IMZ4YwjhGuD7wDjwbzHG/2tLiyVJkgqmlGVZp9swKXMsu7ici1Bc9l2x2X/FZd8VW6UyUFrM+S58KkmSlJiBS5IkKTEDlyRJUmIGLkmSpMQMXJIkSYkZuCRJkhIzcEmSJCVm4JIkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJGbgkSZISM3BJkiQlZuCSJElKzMAlSZKUmIFLkiQpMQOXJElSYgYuSZKkxAxckiRJiRm4JEmSEjNwSZIkJWbgkiRJSszAJUmSlJiBS5IkKTEDlyRJUmIGLkmSpMQMXJIkSYkZuCRJkhIzcEmSJCVm4JIkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJGbgkSZISM3BJkiQlZuCSJElKzMAlSZKUmIFLkiQpMQOXJElSYgYuSZKkxAxckiRJiRm4JEmSEiu38qAQQgn4JfCz+q7vxRg/3HTOBcDrgAnggzHGu5bSUEmSpKJqKXABxwF3xxjPmu1gCGEH8ArgJcAxwPXAi1r8XpIkSYXW6pDiC4GjQwi3hRC+FkIITcdPAW6OMWYxxp8D5RBCZUktlSRJKqh5K1whhHOBDzXtPg+4KMb4xRDCKcDVTK9grQeebPh6H7ABGJrjW5UqlYEFNVork/1XXPZdsdl/xWXfHTnmDVwxxiuBKxv3hRDWks/NIsb4nRDC0SGEUowxq5+yF2h8FQ0Ae9rTZEmSpGJpdUjxAuCDACGEE4CfN4QtgDuBM0IIXSGEY4GuGOOupTVVkiSpmFqdNP8PwNUhhMmrEP8AIIRwMfClGONdIYQ7gO+Rh7rz2tBWSZKkQiplWTb/WZIkSWqZC59KkiQlZuCSJElKzMAlSZKUWKuT5tsqhPAG4M0xxrfVvz4JuIx8Qv7NMca/6WT7NLeF3OpJK08IoQv4NHACMAa8J8b4QGdbpYUKIdwDDNe/fDjG+K5OtkcLE0J4CfCPMcZTQwjPAa4CMuA+4LwYY62T7dPcmvpvB/BfHHrv+0yM8T8O99iOB64QwmXAGcAPG3Z/FngT8BDw1RDCjhjj3Z1onxZkzls9acV6PdAXY3xp/UPOJcDZHW6TFiCE0AcQYzy1w03RIoQQ/hx4BzBS33Up8Fcxxm+FED5L/vt3Q6fap7nN0n87gEtjjJcs5PErYUjxu8D7J78IIawHemOMD9bX9roJOK1TjdOCzHerJ61MpwDfAIgxfh84sbPN0SKcAKwNIdwcQri1Hpi18j0IvLHh6xcCt9e3vw68etlbpMWYrf9eF0L4dgjhyhDCnLcNWLbAFUI4N4RwX9OfF9XLb41rU6wnX6l+0uRtgbQCzNaPwGPkt3p6JfD35Ld60sq3nkNDUgDVEELHq95akKeAj5GPDrwPuMa+W/lijNcD4w27Gu/Q4nvdCjdL/90F/FmM8eXkI3IXzPX4ZfsFne0WQYfhbYFWsBZv9aSVqfl3rSvGONGpxmhRfgo8UP8d+2kI4UlgG/CLzjZLi9Q4X8v3uuK5IcY42Wc3AJfPdfJKGFKcJsa4FzgYQjiuPhn7DOCODjdLc5vvVk9ame4EzoSpC1Xu7WxztAjvJp9zRwhhO3m1cmdHW6RW3BNCOLW+/Vp8ryuam0IIL65vnwb8YK6TV2oJ+n3ANUA3+VWK/93h9mhus97qSSveDcDpIYTvAiXAq9yK40rgqhDCd8inZLzb6mQh/QlwRQhhDXA/8KUOt0eL837gn0IIB8mn1vzRXCd7ax9JkqTEVtyQoiRJ0mpj4JIkSUrMwCVJkpSYgUuSJCkxA5ckSVJiBi5JkqTEDFySJEmJ/T/v+7OxHO+0GwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num3, mu3, var3 = nums[2], true_Mu[2], true_Var[2]\n",
    "X3 = np.random.multivariate_normal(mu3, var[2], num3)\n",
    "plot_cluster(X3, [1, 7], [6, 2], 'r')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第四个例子\n",
    "$$\n",
    "u=(9 \\quad 4.5), \\Sigma=\\left(\\begin{array}{ll}\n",
    "1 & -1 \\\\\n",
    "-1 & 3\n",
    "\\end{array}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlwAAAHUCAYAAADvF/aYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZhc9X3n+8+pvaqrqqu71WgBJECIg2OxmMXYeGNxxmusBCa2cOLYTogzuTcTbnzzZEjseXJzH+6TZGaSDM5kkuttvMUojkWuHMdO7Ai8YEAGjLBx4CCE0IKkptWt6q59v390neJ0dVXX0nXUXdL79Y+666yl48f68Pt9z/dn1Go1AQAAwD2e1b4BAACAsx2BCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlvm53NE3zBkl/alnWTaZpXiPpHyUdqG/+a8uy/s6xb1jSlySdJykl6YOWZU0P7rYBAACGR1eByzTN35P0AUmZ+kfXSPpzy7L+rM0hvynpJ5Zl/V+mae6U9HFJd630ZgEAAIZRt1OKByXd5vj9WknvMk3ze6ZpfsY0zVjT/m+U9M/1n78p6a0ru00AAIDh1dUIl2VZu03TvMjx0Q8lfdqyrCdM0/yYpD+U9LuO7XFJc/WfU5JGO12jVqvVDMPo6qYBAABWWU+hpesarib/YFlW0v5Z0l82bZ+XZI96xSQl1YFhGJqeTvV5O1htk5Mxnt+Q4tkNN57f8OLZDbfJyebJveX1+5biv5im+dr6z7dKeqJp+w8kvbP+8zskfb/P6wAAAAy9fke4flPS/zBNsyjppKSPSJJpmt+S9G5Jfy3p86ZpPiSpKOn9A7hXAACAoWTUarXVvgdbjaHV4cXQ+PDi2Q03nt/w4tkNt8nJWE81XDQ+BQAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAl/m63dE0zRsk/allWTeZpnm1pL+UVJFUkPQrlmVNNe3/pKS5+q+HLMv68IDuGQAAYKh0FbhM0/w9SR+QlKl/dK+k/2hZ1n7TNH9D0n+S9FHH/iFJsizrpoHeLQAAwBDqdoTroKTbJH2x/vtOy7JOOM6Rb9r/KkkR0zS/Vd/+B5ZlPdrpIpOTsS5vB2sRz2948eyGG89vePHszh1dBS7LsnabpnmR4/cTkmSa5o2SfkvSm5sOyUr6b5I+LWmbpG+apmlallVe7jrT06nu7xxryuRkjOc3pHh2w43nN7x4dsOt17DcdQ1XM9M03yfpY5LeZVnWdNPm5yQ9b1lWTdJzpmnOSNoo6Wi/1wMAABhWfb2laJrmL2thZOsmy7JeaLHLr0r6s/q+myTFJZ1osR8AAMBZr+cRLtM0vZI+IemIpPtN05Sk71qW9YemaX5B0sclfUbS50zTfEhSTdKvdppOBAAAOFt1Hbgsy3pR0uvqv4632edXHL++v//bAgAAOHvQ+BQAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJcRuAAAAFxG4AIAAHAZgQsAAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAAAXEbgAgAAcBmBCwAAwGUELgAAAJf5ut3RNM0bJP2pZVk3maZ5qaTPSapJelrS/25ZVtWxb1jSlySdJykl6YOWZU0P8sYBAACGRVcjXKZp/p6kT0sK1T/6c0kftyzrTZIMSTuaDvlNST+pb/+CpI8P5nYBAACGT7dTigcl3eb4/VpJ363//E1Jb23a/42S/nmZ7QAAAOeMrqYULcvabZrmRY6PDMuyavWfU5JGmw6JS5pbZntLk5OxbnbDGsXzG148u+HG8xtePLtzR9c1XE2qjp9jkpJN2+frn7fb3tL0dKrP28Fqm5yM8fyGFM9uuPH8hhfPbrj1Gpb7fUvxSdM0b6r//A5J32/a/gNJ71xmOwAAwDmj3xGu/1PSp0zTDEh6RtJXJck0zW9Jerekv5b0edM0H5JUlPT+AdwrAADAUDJqtVrnvc6MGkOrw4uh8eHFsxtuPL/hxbMbbpOTMaOX/Wl8CgAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAYM3Yk5zVHYcPaE9ydrVvBRgo32rfAAAAtl1zM9qbnpck7UiMr/LdAIND4AIArBk7RycW/QmcLQhcAIA1Y0dinJEtnJWo4QIADD1qv7DWMcIFABh61H5hrSNwAQBWRa1W03y1otlKRclKWfOVisIej6Ier6IejxJen+Jeb1fnovYLax2BCwBwRmWrVe3PZfTjXFbz1cqy+054fbo0GNKlwZA2+vzyGEbL/aj9wlpH4AIAnBG1Wk2P5TJ6JJNSoVaTJEU8Hk14fRqrj2blq1VlqlWlqxVNl8uaqZQ1k01rXzatEY9H14ajek04oqCHEmQMFwIXAMB1lVpN/5Ka09P5rCTp4kBQ10ei2uIPyGgzalWp1XSsVNTBQl7PF/NKVir6XmZeP8ym9ZpwRNdGoooQvDAkCFwAAFflq1X9f/OndaRYkN8w9K54QpcFwx2P8xqGtgSC2hII6uZaXC+Wino0k9LRUlGPZNN6PJfRDZGobohE5W0T2oC1gsAFAHDVt1NzOlIsaMTj0e2j49rgD/R8DsMwdHEgqIsDQR0rFvVoNqUXigU9lEnpmXxOb4sldEGg9/MCZwpjsQAA1/xbPqdnCjkFDEN3JNZ1HbaW66t1QSCgf5+Y0PsSExr3+jRTKevLyVP6ViqpfLU66K8ADASBCwDgivlKRf+ampMk3RyNa9zX/aSK3Vdr19xM2322BIL60PikXh+JyiNpfy6rz85O62ixsNJbBwaOKUUAgCseyqSUr1W1NRDSlaFIT8d221fLZxh6UzSuy0Nh/UtqTsdLRf1dckZvicZ1XXikbUE+cKYRuAAAA5epVvRsISdD0i3ReM/Bp9e+WpM+v96fmND3Min9MJvWg+l5HS8V9fZYghYSWBP4XyEAYOB+ksuqXKvpkkBIYz1MJa6ExzB0UzSunx8dU9AwZBXy+uLpU5oul87I9YHlELgAAANVqdX0ZG6h39Y1kZEzfv3LgmF9YGxS63w+zVbK+vLpUzpMXRdWGYELADBQx0pFpaoVjXt9uqiPFhCDMO7z6ZcT63R5MKxCraavJmf1TD63KvcCSAQuAMCAHS0VJUmXBIKrWrQe8Hj0c/GErguPqKKavj5/Wk9kM5KWbzsBuIGieQDAQNltGS4MBFf5ThYapt4cjSvi8ep7mXntTc8pU63ovuQpPZBJSRKLXuOMIHABAAamVKvpeKkkQ9IFqzSd2MwwDL1uJKrHMil9aW5GVj4nnwyNezza6l/9UIhzA4ELADAwx0tFVVTTep9f4TXWjuHhXFqHigUZkmq1mmarVT1fzK/2beEcQeACAAzMbKUsSVrv86/ynSxlN1G9dSSuH+YWarleFQyrVqvRIBWuI3ABAAYmWalIkhJeb0/H7UnOatfcjHaOTrhWU+VspvqzxYJ2z82qXKvpX9PzemsfzVmBXqyt8V4AwFCbr49wxb29/fe8vXbi3SeP9PzmYD9vHG4JBHXb6Lh8hqEncxl9J5NSrVbr6bpALwhcAICBmauPcI32OMK1c3RCE16vZiqVZResbqXVQtfdhLCLAkHtiI/JI+mxbFqPZNM9XRfoBVOKAICBmavWA5ent8BlT/XZ04q9aLXQtR3CnOduZWswpHfHx/SP86f1UCalqMerK8O9LbQNdIPABQAYiFqtply1Kkka6eMNxV4XrF7uuFYhrJ3LQ2EValX9S2pO30olFfV4dEkw1PN9AMvpO3CZpvkhSR+q/xqSdLWkDZZlJevbPyHpDZJS9X12WJY11/edAgDWtHL9T59hrHoBeq/h7arwiJKVivZl0/rH+dPamVin9f6196YlhlffgcuyrM9J+pwkmab5V5I+a4etumskvc2yrFMruUEAwHCo1IvOnZOJZ+Ltw0F580hM85WKninkdP/crH55bJ1iPdaiAe2suGjeNM3rJL3asqxPOj7zSNom6ZOmaf7ANM1fXel1AABrW9kOXI7RrVYF7WuVYRh6RzyhC/wBpaoVfXVuVoX6FCmwUoOo4foDSX/U9NmIpL+U9Oda+I+dB03TfNyyrB8vd6LJydgAbgerhec3vHh2w22tPD9/qaSRXEBxn69xTx+pnq/ASa8+tGHDmrnPTn59YkSfOXFCM6WSvlPL6f3r1svj0hTpsPydYOVWFLhM00xIutyyrAebNmUl3WtZVra+3wOSrpK0bOCank4ttxlr2ORkjOc3pHh2w20tPb/ZclmZbFEBb7VxTzd7Irp50yWShuv/49/uG9GXUqf0VHZOoVxFb4nGB36NtfTs0Ltew/JKpxTfLOlfW3x+maSHTNP0mqbpl/RGST9a4bUAADgjEl5fo0fXvmxaBwqsuYiVWemUoinphcYvpvlRSc9blvU10zT/VtKjkkqSvmBZ1k9XeC0AwBoWqE+7FWpnR93ThYGg3hKN68H0vL4xf1rrPD59PZ0cihcAsPasKHBZlvVfm37/c8fP/0XSf1nJ+QEAw8MOXKU1vETOnuSs7j11QpJ017qNHYPTdeERHS8VZRXy+pvTL8uqj3QRuNArlvYBAAyE3zBkaCFwVddo6No1N6OnC3k9Xch39eakYRh6eyyhCa9PlwVCuiIU1vvihC30jsAFABgIwzAao1zFMxy42q2d2Pz5ztEJbQ+GtD0Y6noJoaDHox2jY7oiHNHbYgltpgs9+sDSPgCAgQkYHhVqFRVrNbkRS9o1Um21duKe5KzuPnlEM/UFte3u8/1MB67z+fW2WEL/OH9aD6bntcHv1yZ/oOfzfPz4Ee2en9Xt8XH9v5Ov7vl4DC9GuAAAA7PSwvl2I1X2trtPHmnZSHXn6IRujcaXLGA9U6lowuvteUHsVtf+v6eOqVStqqKa9sydVqa+UHcvds/PaqZS0e75pd8PZzdGuAAAAzPi8WimImWqVU32cXyrkSrntnYBqtMC1istcrfvq1ar6edGx/VSqaivzyf1i6PjPTVFvT0+3hjhwrmFwAUAGJio1yuVpFSl99EfaXFIWm5bNwGql+nDVlOVzs+c174lNqrPn57W4WJBD2VSenMPTVHv2bRZ92za3PX+OHsQuAAAAxPzLCz2nO5juk1aPiR1E6C6XSy7eb9WI2vOz+7bsm3R+d4TH9PfJWe0L5vWpcFQX/VcOLcQuAAAK2YHmBvCUUlSqs/AtVLLTUkut1+rkbXlRts2B4K6PhLVD7NpfXM+qQ+OT8rn0nqLODsQuAAAK2YHmGy1ojeMxJXsc0pxpZYLScvt5xzVsn/vNKL2xpGYDhbymqmU9YNMypX1FnH2IHABAFbMDi7viiZ0vFLSvEuBq9OUYbd1W63263Z0zPZPc6f19fnTGvf6ZEi6LBjSRqYW0QZtIQAAK7YjMa77tmzTe8cWgtdcpeJKt3k7FHXTJb5XrVpLdLqXH2TTOloqqibpG/NJlddoh32sPgIXAGBgAh6PYh6vKqq5Mq1oh6Kt/uCifl2t+nc1f9bpdzs0dvtmo30vvzF+nsa9Ps1Uynokkxrk18VZhMAFABioSd9CtcqpSqnl9uWam3Zih6KDpcKika5WI1/Nn3X6vdd7s+/l9rEJvT2WkCFpXzatk6Viz98LZz9quAAAA7XO59cLxYKmy2VdFly6vddaqVbsaT97pGurPyg1TQc2F8Z3+nMl93ZBIKBrwyN6PJfRN1JJfXBsUl7eWoQDgQsAMFCNEa5y6xGu5rDUTyd4u+j9jsMHFgJSNK77tmyT9Eph/VZ/sOUx7X533ls/SwG9KRrXwWJBp8plPZJN640jsZ7PgbMXU4oAgIFa5/VLkqbL5ZZTdO2mBbvRfL52ayjuTc9r9/xsz+dvVcfV7TSj3zD09tioJOnRTEpTpdaBE+cmRrgAAAM14fPJI+l0paxvp5J6sF5I3mo0aapU1FSpqD3J2bajXM5WEM1Tfq36ZzlH0A6WCisaSbPP3e0044WBoK4Nj+iJXEbfTCX1gbF1TC1CEoELADBgPsPQWP2tvXfGEvIYRsspuh2Jcd176oSeLuR176kTbcOMM/B0U3fVPFVoTztOlYpdLfvTbKs/qP1e75IpynbeNBLTwWJeL5dLejSb1qlSqa/r4uxC4AIADNx5Pr9mKmW9JhLVByfO6+qYdk1Nmxet7rXuyv58qlTsqyD+YKmgmUpFB0uFrvYPeDx6eyyhXckZPZJJ6eFMSg9l0z1fF2cXAhcAYOA2+v16ppDTsVJRV4Yjbfe7a93GttOFtuaQ9fHjR7R7fla3x8d1z6bNy3aX35Oc1b2nTkiSbozEtN4f6Lkgvp8i/82BoF4THtGTuYwuCgQVaDPKh3MHgQsAMHAX1Je4ealDT6pWYalTsNk9P6uZSkW752d1z6bNi7Y1j5LtmpvR04W8JGm9P9B4k7EXS96IVHcjVW8ZiemFYl5SQL+YmNDreWvxnEbgAgAM3Hk+vwKGodOVstKViqJeb8djug02t8fHGyNczexRMrtea6s/qKlgSNLyrR6crSQOlgotw16vLSMCHo/eFkvoK8kZPZpNa3soolgXfw84OxG4AAAD5zEMbfQHdLhY0EulokxvuOtjOwWbezZtXjKyZU8zXh2M6NZo/JV6rWhcD1z66o7XtIPafq9XM/UliZoDV7cLYztdFAjKDIZkFfL6XmZe74qP9XQ8zh704QIAuKLbacVmndY0bNUXy55m3F/I6r4t23RjJKaJHt4stPt5XR2M9HRcN948EpdXhn6az7HszzmMwAUAcMX59cB1rIuQ0a65aKvPW62BeHt8XBNeb2Oasdc3C+2QJ4/R03HdGPP5dE1kRJL0YHpetVptYOfG8CBwAQBcscnnl0fSy+WSitXqsvu2ClGtPt+TnNVUqajtwdCiKcd7Nm3WM5dfresj0cbais0d6O3jl+sa36pz/SC8PhJV2OPR0VJRz9WL+HFuoYYLAOCKgMej83x+nSyXdLxc0kWB9tN07eq2mj+33zq8NRpvOeXYaC3hWFux5Xa1Lsjvp06rlea3JUMej944EtO3U3P6bmZeW+uF/Dh3MMIFAHDNBYH+6rhszfVcW/3BZWusOo1Q2dvt1hPtRrpajYTZn338+JGOayu2GrG7KhTRhNenZKWiH+UyHb97t2s4YjgQuAAArrmgyzqudlOKzZarzWrXqd6p24WzW91PL4titwp+HsPQzdG4JOmRTEqZ+tuQ7XT7d4LhwJQiAMA1m3wLgetEqahKrdZ2Iedue1wt1/W90dqhPnq03NRgq+s5A1ur7c2LYi93r+2mJi8JhnRxIKhDxYK+k0zqtQr0dI8YXsYaeluiNj2dWu17QJ8mJ2Pi+Q0nnt1wG4bn9+mZlzVbKeuXxtY13lxcKbs56q31Wi17CZ8XiwWlazVNeL36kw0LvbraNTR1LvtjLzG0Nz2v7cHQoiWABr3w9HS5pM/NTis6EtQvBuNa5/MP5Lw4syYnY63/66ENRrgAAK66OBDUbK6sA4V828DVzXSgc9/mNxXtYvrtwZBOlEuaqVQaU3F70/PaZxhK12qaKhUXjYjZy/7cffKIbo+PayoY0ovFYuNz+3hpZQtPN3+/q8IRHaiV9Z30vP59ghGscwGBCwDgqm3BkJ7IZfR8Ia+3jMRktJhW7PT2YPO+zW8qtnqb0TkVZ+VzSpdLi86z1R/UPsNQqaZGXdh6f0BPF/KKGh5ZuawkLWlB0Y/m7/eGkZiO5Of0QjGvFwp5XcJbi2c9AhcAwFUX+AMKezx6JJPSN+dP61fGJle0TmGrfZtrppp/dq6VaNd+HSwtTD82TyFK0lTplVEuMxxZ8XRi8z2PeLx6cyKhPz11SB+cP6iPTmzQL4wx0nU2I3ABAFzlMQxdGgjpS7PTOlQqKuDxdL1Oob1G4u3xcV0fiTZGrlr12FpOq4WxnSGoVUCz67sGUbTe6vvdEIvp2UJOB4oFffr0NIHrLEfgAgC47rJgSNtDEQU8np4CjL1G4u752UYrB2nptKOzRkpaXOje/PbhVKmoqXqbCrvg3h71ch7rXPS6lxqzbo/xeTz6pcQ6ff70tLb4A8pXqwp56NZ0tiJwAQBctyUQ1BXhiMxQWDfF4l0fd3t8fNEIl9R6xMlZI2VPB9oF8s5t923Ztqi/VfN2SS1bS/RSY2a799QJPV3Iy8rn2gav31y3XgmfT8dKRe3LpvWW6NK/m37CHtYeAhcAwHU+w9AlgdDCFFohr+vq4amTezZt1j2bNjd+bxc4tvqD2l/vQD/V1GS1uX6q3Z9b/UE9nE0pahiNtxzbFeX3EoKSlXLbsGbUm6F+8fQpPZ7N6JrwiGJe76J9+gl7WHsIXACAM+LS4ELger6HwNUtZwd6u6fWztGJRbVYNmc9lTM4OVtLSAsjZXuSs439nWGnmxBk30enRqkb/QFdHgzr2UJOP8ymdWtsdNF2GqCeHQhcAABXNI8CXRIIyitDR0tFZaoVjXi8nU/SpeYCeDsE3XH4wKJeWzZnyNqbnm+Mim0PhhY1QXWOcrW7Xju9LIT9upGoni3k9FQ+qxsiUUUdo1yDWlAbq4vABQBwRfMoUMjj0eZAQIeKBb1QKOiKcGTF13CGulZvLtpF8i8WC0uaodrbpVfqvpy9vZzb7Ws5O9P3+qZks69MTemTR19qhMTLgiE9V8jrh7m0bomOdj4BhgqBCwDgilajQJcFQzpULOi5Qn4ggavT1J49OtT8FqN9X+22txpVcnambzfy1Uq7eq/PnTy56N5fH4npuUJeT+Wyem148SgXhh+BCwDgilahZWsgJENzerFYUKFaVXCFbRC6rW9qFayae291ClD2aJm0UGB/y/M/VbJcVsLn013rNrY9vl0o/NCGDSqWKo17X+/3a1swpAOFvB7PZXRTizcWMbwIXACAMybq9ep8f0DHSkUdKhZ0eSi8ovO1C0rtgpXdqsG5pmI3x9nXkhYC1MPZVGO061ilvOyIV7tQ+N7163WzZ/Eo3+sjUR0o5PVkLqPrIyMDrXPD6iJwAQDOqEuDIR0rFXWgkNfloXDffaaWO84eVWrup2V7sfjKG4jOc9m1XPsyKd176oTuWrexcT5nkX3U8OgCb/2fUMNY9EZjs26L3u17qNVqOlQsqFCp6u4N53f994G1jZa2AIAz6rJ624UXinmVa7VFbwT2Yrnjdo5OKGp4NFOpLGoLcde6jQpKSteq+qOTR5ecS5KihkfpWk1PF/K699QJ3X3ySOM6O0cnNOH1Kl2rygxH9KPLr5IZCuvpQr7n+29276kT2pue16OZlA6VitqTOq1stbqic2LtYIQLAHBGJbw+Tfr8mi6XdKRY6LoOS9KSZXraHbcjMd6YPmwezSrX90k6wozzXPZxUWNhTGKmUpFPC3VbzmnFdg1UV2rc59dFXq+2BkJ6MpfRG0ZiAzkvVteKApdpmk9Kmqv/esiyrA87tv26pN/Qwv+277Es6+sruRYA4OxxWTCk6XJJzxbyPfWZal6mZ7nj7lq3Ub/10qGF0aypY41lfCpa+Mfv/YlXAlKrxqZ2gLr75BHNVCr6cvKUHs6mlrSEaNdItdfeWc6GrddERnRfckZPZDO6PjyiAGssDr2+A5dpmiFJsizrphbbNkj6bUnXSQpJesg0zW9bllXo93oAgOHmDCNvisb1g0xKViGnW6rxlos2twovztGkTuFmR2Jcv3P8RRVqNSUr5SXHtwtEzQFqo8+vQrXamGZsVyC/JznbCGf2eXrhvG6tVtMmf0DHS0X9JJ/TtZGRns6FtWclI1xXSYqYpvmt+nn+wLKsR+vbXivpB/WAVTBN83lJV0p6bLkTTk4ybDrMeH7Di2c33Ibl+d1//AXtTc8r4Pfqzm1btN0o6FAup5NhQ9fHl36H5v0l6c7JmO7Uws/vfOqpRdu/MjWlz508qQ9t2KD3rl8vSfq1TZv0t1NT+qX16/VgNav7s0l95MLzG9u7ueenC3nFPB5t8fs15vfrIxeev+jv/CtTU/rjw4d1MJ9XqlrVOp9vyT7tLLfPO0a8+s+HDul3Th7WH27Zop0bNnR1z1ibVhK4spL+m6RPS9om6ZumaZqWZZUlxfXKVKMkpSR1bJs7PZ1awe1gNU1Oxnh+Q4pnN9yG6fndFkmoWKrotkhC09MpXVzy6OlsUQ8WT2lLfmEh5+X2b3W+o9mcjmZz+vSBw43pxmKp0mi38LHEBm1XQPeeOqHPHD+hdK26aPty9iRndTSbU9TwKFWt6rWBUGMqcXo6teTNRkma8Hr1x+sv1M2eSMfn0unZjddq+vF8Ws8V8/qLw0d1q5dRrrWk1//QWUngek7S85Zl1SQ9Z5rmjKSNko5KmpfkvJOYpOQKrgUAGHLNdVLbgiGFPR5Nl0s6WS5poz+w7P6tzmeHrLtPHtHt8XEpGgJPyuAAACAASURBVF8y3ejsED/h9S5b3N7quAu8PgVlaKs/uGQxbHuxa3vB6+UaoDZf5/7jL+i2SKLt/oZh6H2j4/pS8pS2BUKq1WpLQimGx0oC169KukLS/2aa5iYtjGrZ/yv8oaT/p17nFZT0KklPr+RGAQBnF59haHsooseyaT2Vyy4JXN3YOTqh/bmMZioVHSwVGiNQdxw+sGi9RLtDfKdA5CzKt4OZlctqplLRw9mUDpYKjfC2PRjSrfWA12u9lnM0brljf3Nyg7wej9LVig4VC7qkHuwwfFby2sNnJCVM03xI0t9pIYD9tmma77Es66SkT0j6vqQHJH3Msqz8iu8WAHBWuTK0MLX3bCGnQh89p3YkxvUnGzY3go9t5+jEojB017qNWu8IdHuSs7rj8AHtSc4u+myqVNT2YKhx3M7RiSXtI+wRLefbis3n6mTn6ITeMTbWsZWEzzB0Xb1gfl823fK+MRz6HuGyLKso6f1NHz/s2P4pSZ/q9/wAgLPfhM+nzf6AjpSKeqaQ09Xh3uuUWk09tmrzYI9cSVr0NqG00HT0xWJB6VpNt0bjjXUXf+f4YaVrVUUNT6Pr/Hp/YNGoVqcFtNvd853btnRVf3dVKKJHM2kdLRX1SCalH2TTPV0LawONTwEAq+rK8IiOlIp6KpftKnB12+uqXZPUXXMzmqlUGvVc7Wq87j21UGQvSQmPZ1GB/FSp2PLc7e5tJf25gh6PrgpHtC+b1pWhiEIez8CarOLMIXABAFbVZcGQQoZHU+WSTpaK2tChlqvbEaVOTVK3+oMLy/FUa4oaHiU8Hv3hhgslLUwRJssLvbuihqGEz6e96flG3dZUqdjy3HbtmDOQPZZN67Onpxsd7vsJYleHI/phNq1Rn0+/EBxrLCPEKNfwIHABAFaVzzC0PRzW49mMnspnOwaubpuftltyx55utMORvTbiRb7AolGs7cGQzHCkcfyuuRlt9Qd1sFTQjZGFF/GbF62293UGsv25jMpa+Ae3+V66DY+jXp8uCgR1qFjQ55LTejyX7XgM1hYCFwBg1V0ZiujxbEbP5HO6aSSu4DJL2Thrp5w9sOzP7PDVqa2EHX7sEGWHpKhhNIrincc7Q5qica33B7Q3Pa97T51Ycl1nENzqD2r3/Kxujy+9n17WYbwqHNGhYkGXBcKKe5Zvb4G1h8AFAFh163x+XeAP6FipqGcLOV3VoZbLHhlytmZYbrTIDkB2uLL/dI6MOZfmWe8PtAxrzpD2cDbV6L/VfF1n2NuRGNc9mza3/B69rCO5NRDSiMejTYGAfve8Tbog0HsbDaweAhcAYE24MhzRsXrxfLvA5QxOauqB9Vg2rf1e78K2JnYY22d4lK5VG386t2/1B7XR59dGn7/tdKUdkG55/qeNaccbIzGdKJdaXneQvIahK0IRPZpN66l8hsA1ZAhcAIA1wQyG9YAxr5PlkqZKJa33+5fs0xjFisYbPbBsB0uFRgPUZs4mpulKVX4tvJFoF87vTc/ru5LKUqMthLN5avMoVLLeUiJZv95MpaLd87O6PhJt3Gc/byR2cmV4IXBZhbxuqVYVXmbqFWsLgQsAsCb4DUOvDoX1RC6jp/IZ/Tt/Ysk+y9U8LbetubbKrv2ypxX3ZVJK12qKGsaS87SslarVGn86u93bbw82B7XmKc2doxO6s49FxxP14vkXiwX9Wz6nayOsrzgsCFwAgDXjynBET+ReKZ4PNI3gLFfz1E09VKui9h2Jcf3RyaNKV8pKeLwt67CaJXw+HauUlfD5FhXxO99SdL7BaI+i7fd6Gw1X79SW7v9iHK4KRfRisaCn8hldE46wvuKQYCwSALBmTPr8Ot8fUKFW07OF/laE62b5mx2J8UX9sxI+36I/O7lr3UbdGo3rxkhMdxw+IOmVflw7EuNa7w/o6UK+MeJlLzV0e3x8yTJEvbo0GFLE49GpclnHy6W+z4MzixEuAEBfVtI9fTlXhSN6qVTU47m0rgiFex7BafW2ovNe7X2c933Xuo2Ltjd/tz3JWd176kRj3+ZeXs5rSUunI3t5G7ETu3h+X33R7/P7WPQbZx6BCwDQl17XEOw2oL0qGNb3PSmdKpf1QrGgrfXWC83nadXaQWpde9W8lqLzZ/uenEX4zd/NufyPs+9Wp+aq/Xz/blxZD1xWIadbqnGFKJ5f8whcAIC+9NK0U+o+oHkNQ9dGRvSd9Lz2ZdNLAlereqjmBqX2qNQdhw+0DUZb/cFG3y3nUjw7EuNL9t/qD2qf4ZFf0ovFV5qttloyqJfv30sIa953cyCoI/Xi+Wsonl/zCFwAgL70Ok3WU1f1UESfnXlZu5MzKlSq+vC68xrbtvqD2u/16upgRPIYS87X/CaitDQY2dOB9iLW0uI3C52F8NJCy4l0rbqwDJBj4etedBp56/R32bzvVaGIjhQLeiqf1Wsonl/zCFwAgDOil4AW9Hh0qFjQoVJRX0ieWhS47L5X8hhLenFJrwSTqOHR9mCobTBqDkDOGi7neZz7qFrT/kJWVwcjPS8g3er7twuhrUa+mvfdFgwp7PFoulzSyXJJG6nlWtMIXACANelXxyb1FzMndUkgqFPlktb5Fhqhdhopc/bFWl8PIfbUYrupR2fYajUN6SySn6lUtL+QbTmdKfU2Tdiu1sue6tyfyyzaz7mvzzC0PRTRY/XieQLX2kbgAgCsSe8bX6cxv0/7c1n9MJvRO+MLjVA7jZQ198XqVDtlb58qFfVisah0raqpUlEPXPrqJdfpNJ1pX7eXlwmc9iRn9TvHDytdq8orNZqptuodJi1MvT6WTeuZQk43V5df9Buri8AFAFizrg9H9VQuq2fyOb1pJKZYvd6qE+fo1VSpuGRqsdV04VSpuGh9xVYBp9N0pvN8/fTa2jU307iHjV6fzHBk0ZTn3vT8olGv76fn9U/zp3VpINTVot9YPQQuAMCq6TT9NubzyQyG9Wwhp8dyad0SHe3p/HY7B3t9RFur6cLmXlvNrSSaF81up/ktyVbtK/YkZ/VXLz6rcrnS6Otl389Uqdi4h+Z7di4hZLereKaQV75alVXIE7jWMAIXAGDVdDP9dkMkqmcLOT2Vy+qGSFQjnu5GuaT2o02tWkc4pyqbR8Ya047BUKMuzN6vXWBsXhTb+R13zc1ofybT+Nl57Hp/oLGotvOYVksI7RydUKVWU9zj1ZFiQTkWtF6zCFwAgFXTzfTber9fWwMhHSzm9Xg2o7dE45K6K07vVO/VLvC1GxmbKhWXNERtFxidI1LNbSR2jk5oplZRuVzRztGJJa0sOvUYa/59V3JGR4oFvVDM69WhSNvvi9VD4AIArJpuW0XcOBLVwWJeT+Yyuj4SVcTjWVFxuq1d4Gu3NE/zG432FKBzoWrnd5OWLiNkn+P3N2/W/PzCeotWLqtjlbIu8Pq0PRhSslxetqVFs22BkI4UCzpQIHCtVQQuAMCat9Ef0MWBoA4VC3oim9abHHVU3YSSdqNhzdOIzfs0T+s9lk1rfy6jrf5g47MT5dKiuiqnVoHSDoqBk14VS5VGzzBpYfFse+FrMxzpOkg+l89pd3JGV4VH9K74mPw0QV1zCFwAgKFw40hMh4oF/SiX0XWRaE+NVFsVwDc3PG21fmLzCNru+VnNVCraPT+rezZt1q65mZZThsuq1hr/+DY3VL0xEtP1keiibd1MnX4tdVqH6sX2LxYL2ta0HBJWH4ELALCmOQPHlkBQh+uh6w0jsa7P4RwNa7eQdfOIWasRtNvj49o9P6vb40u7v7frldVsfyGrsqTHUil9ftMlixqqHiwVdE9i85J6sk5TpztHJ3S6Utb5voAOFPIErjWIwAUAWNOcgeO/btyiw8WCnshmdG14RKEu38hrt6zOY9m09nu9jSnC5QrUJemeTZt1z6bNktpPQXYKR3Zo+6X16xfdi/PP5vtst815r2+IxvTZ2Wl9bW5WX5h9WXck1vVd24bBI3ABANY0u7v7Vn9QFwQC2hwI6kh9lOvGHka5bM2LU9sjS500B6xW4coORVv9wUXtJpzH2qFtcjKm6enUknta7n6Xs87n14TXp925jA6VijIMg8C1hhC4AABrmt3d3Q5FN0aiOlIs6PFsRq8Jj6yo79Rybxl2CljN4crZ4LS5K3zzsXuSs7r/+Au6LZIYaCi6NBjS9lBE8V5qynBGELgAAGta85Tahf5XRrkeyaR0S6y37vPtglTzW4bNIck50mZ/ZtdfORuc7s9ldHt8vNFL6+6TR3R1MKIJx7H2uY9mc4u620ta9HuvYeyyYEhmKKzrIlG9Z3RsRX8vGCwCFwBgTWueUjMMQzePxPWF4rR+lMvo6vCIvl8PTJ3Cwp7krO4+eWRRU9FOvbjsEaypUrHl9OPO0Qnty6SUrtUaC04fLBX0Jxs2N661v5BddOzO0QkF/F4dzeb0dCEv6ZUWFM7few0++9Ip7Zmb1eXBsE6Ojmmjoyt+J4Poa4b2CFwAgKGz3u/XFaGIfpzP6ruZeX2ly7DQqo1Duxop+zM7NG0PhnRri3UUdyTGde+pE3q6kG8sOG0vzXN7fFwHS4VF0432MXdu26JPHzjcGNFyLqLt/L0Xfzc/qwPFgsq1mg4U8j0FrpUsuo3OCFwAgKH0xpGYni3kdKCQ16THt2jKrp3mNg7LcY6GTXi9y07x2Ytd2+e1pxkVjeu+LdtaHvOVqSntmptZcl5nI1Zn4X03do5OKFetasLr04FCXm+uL4PUjV76mqF3BC4AwFCKer26IRLV9zMpPdk0ZddOr81S7bD1JxsW98bak5zVvadOKFleWJY64fMtCk7djBZ97uTJZUfl7FGzqVKx63vekRjXu0fH9FenpjRTKWumXNaEj3/q1wKWFAcADK3jxaL2zM1qxPDoteGRJQHHHiXak5zt+dw7Ryd0azSu2+MLU4a3PP/Txnnsxa2PVco6Vinr6UK+UYMlvVIbtmtupu21P7Rhg26Nxhs1Yu32e7G48AZlt9/l63OntWd+VlY+pwOFXM/fG+4g9gIAhtZX6zVLF/sD+pXxSb0jnli0fSWF4M63EJsL2bf6g9pneOQ3DI0YhhI+35Kw12mE6r3r1+tmT+SV6Uctnk6UpKhhKF2r6u6TR7TR52/cR6c6tZ/kc0pXKjpQyOt1ffQqw+ARuAAAQ2vn6IRqtZrO9weUqVa1r76wta25lUO/12guZD9YKihdq+rWkaU1WnZ7BXu6sZvzO/+UXhlB2x4M6cXiQh+ysIyWRfutzlet1RTzeHWyXFKhWlVwBb3KMBgELgDA0LJHoV4qFfW3p0/psVxGV4VHFPd6JS1tmrqSazgtV6Nlj6ptD4ZkhiOLFqF29ti6czK26PzOInnn+e2RsoTP17YAv9X9fnF2WifKJZ0ol3RRoP/AicEgcAEAht75/oAuD4b1bCGnP3v5uJ4r5pcEl3b6afjpDGHNxzdf067tsoOT/dmd2rLonM7pz/u2bFvShNX5Hbq55/MDAZ0ol/RSqainshmamq4yAhcA4KyQqZR1/9ysUpWypuuNTZuDSyut6rx6CWHNy/g4w5izPuul4sK0pFe9LVLdaoStm9q0C/wBPa6MjpYWXiygqenqInABAM4KX0sl9UKxoFi9kP0S3/JNP+1QtdUflJpqo5wh6rFsutG01J76+6OTRzVdKWvS55cZCMmnhQ7zzd3hnSFqXybd9h7sczdPLbYLR92M3J1fb3p6olTUe+PjHfeHuwhcAICzgl0s/m/5nKYqZf24Q0uExihRi+akO0cntD+X0Uyloi8nZ5SuVRtvG+6am9GxykJB/LFySblaVWVpUfd6m3N0atfpU9qbTamiV6YUW41UdTN61U0/sRGPV+Nen2YrZb0uGtMvjBG2VhOBCwBwVrBDyL0vn9CXk6d0USCocq0mn2Es2m+5kS1J+vjxI9o9P6urgxHJY8jKZZWuVBvbd45O6EeZtJK1qhIe76Lle3bNzSwZEWvwLNyH3RH/nU891fIeBrnEzvn+gGYrZR0rFXta5geDR+ACAJxV/uPkBsV8Xp0ql/V4NqPXjUQXbV9uZEuSds/PNhacfubyqxdN+0lqjHLtTc/rmsiI7tm0WdIr9Vr7vd5Fi2NLCyFvqlTU9mCosQzQ3vS8bm1xD4NcYucCf0A/yWf1Uqmo6wdyRvSLwAUAOKt4DEM3R0f198kZ7cumtD0UVrTeJkLqPIJ0e3xcu+dndXu97qnbthD2z1v9QT2cTWmqtNAh3g5oTxfyujUab5wr4PfqtsjiRq2DZtdxHSsVVavVZBhGX29lYuUIXACAs87FgaAuCQT1QrGgB9Lzes/oWGNbpxGkezZtboxaNXOGlXYjU3uSs41RMruIvjmg7UiM685tWzQ9nVrpV132Ht8zOqaIx6NstarTlYrGfb4Vdd9H/whcAICz0s/GRvW/Zqf1bCGny/IhXR4Kr/ic3YSVe0+d0EyloqhhLApY/YYbZ81Zy9qwDvd4gT+g5wp5vVQqatyxBBFvLJ5ZfQUu0zT9kj4r6SJJQUn3WJb1Ncf2j0r6NUnT9Y9+w7Isa2W3CgBA90a9Pr0lGte3U3P61/ScLgwENOLxdj5wGb2ElYsCwa5CVnONmLMbvbNerFVtWDf3aAeuY6WirghHlm3aCvf0O8L1y5JmLMv6gGmaE5KelPQ1x/ZrJP2KZVlPrPQGAQDo19WhiKxCXkeKBT2QmtfPOaYW+9HNSJVdFN/tCJJzRErSkoWy7bUck5WKtgf9Hc/bfI/224lT5dKy1yZwuavf1Sz/XtJ/dvzevELntZJ+3zTNh0zT/P0+rwEAOEfZzT/3JGdXdB7DMPT22KgChqFnCjk9t0xvrkFdc0divKsO97adoxONRam3+oOKGoYu8PkXTUeu9wd0rFzSen+g43mbv8c678LYymylrGqt1vbacFdfI1yWZaUlyTTNmKSvSvp40y67JP2VpHlJ/2Ca5rsty/p6p/NO1hfyxHDi+Q0vnt1wOxuf3/3HX9De9LwCfq/u3Lal8wHLmJS0I+LVN2Zm9HA1r9eMTyjiXTq1OMhrdn1vkzHdORlrrKt4/1NPKV2r6U2x6KJ7+Ej1fAVOevWhDRs6Pu9W3+P84ryS5bI8iZAmA6/043JeG+7qu2jeNM0LJf2DpP9pWdaXHZ8bkv67ZVlz9d//SdJrJHUMXG68rYEzY3IyxvMbUjy74Xa2Pr/bIgkVSxXdFkkM5PtdVJPGS9LRbE67XnxJPxdfOrU46Gt2qo96sJrVJ4++tGh7u3u42RPRzZsukbT438pW12h1jlC+qkyxqGenktIAXh5A7/+h02/R/HpJ35L0W5Zl7W3aHJf0tGmar5KUkXSLFgrsAQBYFQtTiwn9pxNHtDs5o5cKBf2HyQ2L9ullLcNudKqP+tzJk0u29/o2Y6trNJ9jT3JWn5yZ0gX+gKYjUV0uAtdq6HeE6w8kjUn6z6Zp2rVcn5I0YlnWJ03T/ANJD0oqSNprWdY3Vn6rAIBzhRvF3GM+n46VCjpUKuoLyVP64MR5CnuWljK3una3b/M5WzjYneXb1Ud9aMMGFUuVFdVPdfPW5K65GT2ZzypZKetUpbnkGmdKvzVcd0m6a5ntX5T0xX5vCgBwbnOrV9RHxs5TpnpSlwZCeiA9p3e1mFpsde17T53Q04V8YwHrdppbODg7yzd77/r1utkTWdH36WZEbOfohIrVqhJen6ZbvKmIM4PGpwCANWeQ6wk6/fzYhN4cG9XnT0/rp/mczGBYlwZDA7u2c3kfe0HrQUxPrsSOxLjePTqmv5g+oWSlolKtJn/Tgt5wX79tIQAAGErjPp/eOLJQ8PwXL5/Qe198rmMriBsjMU14vbox0l2h9PWRqO7bsk0HSwXtTc9r19zMiu97JbyGodF6e4g5phVXBYELAHDOuTY8ovP9AT2eS+s7mVTHQHSwVNBMpaKDpULjs1Z9u+wpRft8a6nPVbzeCmOu3q0eZxZTigCAc46n/tbid1JzkqSbIvFl929V19VcXL8nObukUN6tqdHltCvwT3i9OiwC12ohcAEAzkkTPp9+fd16fSc9r4oh5arVlm8tSq2DU3MIswvrt9drwtyu3WoXrOwgOFUqLto+Wl9Hcq5K4FoNBC4AwDnruvCIDhTyeqlU1D/Nn9bto+MyuiwoX2706kysUdjuGnYAnCoVF22P1qcU0wSuVUENFwDgnOUxDL07nlDI8OiFYkGPZNNdH9tcw3XXuo26NRrXXes2LqrdGtQajc3a1YfZazna92Nvj9VHuNJMKa4KRrgAAOe0Ua9P744ntHtuVj/IpLTB59clTa0iWmkeYWoe8bJ/vuPwAVdGuzrVhzVvj9qBq1od2D2ge4xwAQDOeZcEQ7pxJKaapK/PJ7tqndDtG4j9vqk46JGxkGdhqjRfI3CtBgIXAOCctyc5q7+cPqHT5bLytaq+Nnda5Vpt2WPsqbvmdQubQ1Kr/Zqv3XzMnuSs7j55ZEU9vJrPGzQW/skvVpf/XnAHU4oAgHPerrkZPZBJ6SZJtwZHdaJc0gPpOf27WKLl/s43BO3jd45O9FUs31gOKJdpHLdrbkYzlYomvN6+e3g134tXC6MsFdVUrtXko9v8GUXgAgCc85wtHm4YierLyRntz2W10RfQFeGl6x06w4ykxs/9rAG5c3RC+3MZzVQq2jU3ox2J8UXnWekyQ/afhmEoYHiUr1VVJHCdcQQuAAAcNvgDemt0VP+cSurb6Tmd5/Nrvd+/aJ9WwcoOR90GpK9MTemTR1/SztEJ/cmGzYtGzAbRMLXVOYIeQ/mKVKxVFaGq6IwicAEAznnN029XhiN6qVTUT/JZfW3+tD4wtk4hR1PUdm8kduKcirw/m2xcc7kar0GyF60udqhPw+ARuAAA57xWI1ZvjY3q5XJJU+WS/mk+qV8YHZNnhdNwzmD3kQvPV7FUOaPrLPq1cP+dXgjA4BG4AADnvFbTb37D0I7RMd19/Ih2J2f0TD6rj60/f9lO9O2W27E5g91716/XzZ6l9WFuslueekX91pnGBC4AAG0kvD6dLJd0qFTUP86f1uP1NwnbsUewmls52C0apDM3fdhKpT6y5SVvnXGMcAEAsIwPj02qUqvpPJ9f30nPa9Tr1WXB8KJ97JGtrf6g1KLJ6ZlYW7EbZS0ELt5QPPMIXAAALMOebnw0k9L3Mil9fT6pnQmvNvkDjX0agSoa131bti05Rz/tItxQqZduMaV45hG4AADowg2RqOaqFT2Vy+r+uVn9UmKdxnwL/4x2ClQrbfPQqTasW+XGlCKB60wjcAEA0AXDMPTW6KjmKxUdKha0e25WOxMTinq9A+mbtZxBTEnWajWV6oHLT+A64yiaBwCgS17D0HviYzrP59dspay/n5tVrtr7YtB7krN651NPdb0wdb8LYDulq1VVVFPE4yFwrQICFwAAPQh6PPrFxLgmvD5Nl0v6anJGhR5D1665GX3z9OmOC1MP8u3GucpCU4hRj7fvc6B/BC4AAHo04vHqvYkJJbxenSiXtHtutqfQtXN0Qu8YG+s4YtWuzUQ/5qplSdKol2qi1UDgAgCgDzHvQuiKe7w6Vipq99ysil2Grh2JcX3jqqs6jlgNYirRNm+PcHkZ4VoNBC4AAPqU8Pr0vsSEYo7Q1ev04nJ2JMYH1ih1pmKPcBG4VgOBCwCAFRjz+bSzHrqOloq6+8QR/eKLz3VdEH8m1Go1HSsWJUkbfYEOe8MNBC4AAFZozOfTHWMTGvf69HAmpe9mUvpicnq1b6thrlrRfLWikOHReT5quFYDgQsAgAFIeBdC1xtHYpr0emXl8/r8zMt9nct+O9E5Stbqs24dqY9ubQ4Ell18G+4hcAEAMCAjHq/+eONmTfr8mqqU9TezL+tQsdDzeVq9nbiSNxYPlxbu4UJ/sOdjMRiMKwIAMEABj0d3TWzQ/5id0kX+oHYnZ/SWaFzXhUe6Hl1qtVRQv+sx5qtVPV/IS5IuDhC4VotRq7f5XwNq09Op1b4H9GlyMiae33Di2Q03nt/aVavV9L1MSvuyaUmSGQzp7bGEgp6FyaUz9ewey6b1YHpeWwJBvS+xuotnn00mJ2M9zc0ypQgAgAsMw9BbonH9/OiYgoYhq5DXl06f0nS51PU5VlK3JUnVWk1P5jKSpGvDI32dA4NB4AIAwEWXBcP6wNik1vl8mqmU9aXTp/STXFbdzDCttNP8s4W8kpWKEl6vLmE6cVVRwwUAgMvGfT79cmKdvpWe07/lc/pmKqmXpmp6nYJKLLPUTr91W5KUrlS0Nz0nSXptJCoPbyeuKka4AAA4AwIej94VS+gdsYRChkcHczn9r9lp7cumVW0x2rUnOatdczPaOTrRc6f5Wq2mf04llatWdVEgqKtCkUF9DfSJES4AAM4QwzB0RTiiS4JBPeEp6dHsrL6bntcz+ZzePBLTxYFg401GezpRUk+Bq1ar6fuZlF4oFhQyPHpHLEHvrTWAwAUAwBk24vHq9smELihI307P6eVySV+dm9UGn1+vG4lqWyDU13RiuVbTN1NJPZPPyZD09vioYqyduCYQuAAAWCWXBEP6sD+gT0yf1P3zs3pVMKyT5ZImvD5dHR7R31xwsUaXqfFyOlYs6juZeR0vFRUwDL0nPqZLgiGXvwG6ReACAGAVBTwePZHP6PliQQmvT9d7opqplLU3Pae9aek8n1+XBkPa4PMr6vEo6vEq5PEoVanodKWsmUpZViGv46WF5XtiHq9uGx3Xer9/lb8ZnAhcAACsMuf04btHx2QV8jpQyOlQsaCXyyW93EXvrqBh6NpIVNeFRxTy8E7cWkPgAgBgle1IjC8qjP+ZUFg/EwqrXKvpaKmoFwp5na6Ula5Wla5WlKtWFfN4lfD5NOb1apMvIDMYUoCgtWYRuAAAWKN8hqGLA8ElayDWajXePBwyRGEAAIYMYWv4ELgAAABcRuACAGCNWOliL5WiTwAACZ1JREFU1Vi7+q7hMk3TI+l/SrpKUkHSnZZlPe/Y/uuSfkNSWdI9lmV9fYX3CgDAWa3f7vJY+1YywvXzkkKWZb1e0t2S/szeYJrmBkm/LekNkt4m6Y9N02SZcgAAlrFzdEK3RuN9LVaNtW0lgeuNkv5ZkizLelTSdY5tr5X0A8uyCpZlzUl6XtKVK7gWAABnvR2Jcd23ZRujW2ehlbSFiEuac/xeMU3TZ1lWucW2lKTRTiecnIyt4Haw2nh+w4tnN9x4fsOLZ3fuWEngmpfk/F+Kpx62Wm2LSUp2OuH0dGoFt4PVNDkZ4/kNKZ7dcOP5DS+e3XDrNSyvZErxB5LeKUmmab5O0k8c234o6U2maYZM0xyV9CpJT6/gWgAADDXeQDy3rWSE6x8k/axpmg9LMiR92DTNj0p63rKsr5mm+QlJ39dCqPuYZVn5ld8uAADDiTcQz219By7LsqqS/kPTx886tn9K0qf6PT8AAGcT5wLVOPewliIAAGdA8wLVOLfQaR4AAMBlBC4AAACXEbgAAABcRuACAABwGYELAADAZQQuAAAAlxG4AAA4h9DxfnXQhwsAgHMIHe9XB4ELAIBzCB3vVweBCwCAcwgd71cHNVwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAMCQ2JOc1R2HD2hPcna1bwU98q32DQAAgO7smpvR3vS8JGlHYnyV7wa9IHABADAkdo5OLPoTw4PABQDAkNiRGGdka0hRwwUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAywhcAAAALiNwAQAAuIzABQAA4DICFwAAgMsIXAAAAC4jcAEAALiMwAUAAOAyAhcAAIDLCFwAAAAuI3ABAAC4jMAFAADgMgIXAACAy3z9HGSa5qikL0mKSwpI+qhlWY807fMJSW+QlKp/tMOyrLkV3CsAAMBQ6itwSfqopL2WZf130zRNSfdJuqZpn2skvc2yrFMruUEAAIBh12/g+gtJBcc58s6Npml6JG2T9EnTNNdL+oxlWZ/tdNLJyVift4O1gOc3vHh2w43nN7x4dueOjoHLNM1fk/Q7TR9/2LKsx0zT3KCFqcX/o2n7iKS/lPTnkrySHjRN83HLsn683LWmp1PLbcYaNjkZ4/kNKZ7dcOP5DS+e3XDrNSx3DFyWZX1G0meaPzdN8wpJuyT9rmVZ323anJV0r2VZ2fq+D0i6StKygQsAAOBs1G/R/M9I+ntJ77Ms66kWu1wmaZdpmtdo4U3IN0r6fN93CQAAMMT6reH6Y0khSfcu1MxrzrKsHaZpflTS85Zlfc00zb+V9KikkqQvWJb104HcMQAAwJAxarXaat+DrcZc9vCiFmF48eyGG89vePHshtvkZMzoZX8anwIAALiMwIX/v727C7WsLOMA/j+TlAhOV3NRUjcGz+XAmFkw2ITFUBJ90U0QlEYoc2NEQRAM3VRKClKUEANe6EWkzEX04Vz05VcNpEFCPKYFGiSUUEZCZp4u9j65PRz3OWfPvLP3rt8PDuy1zlrsB96zWP/zvu9aLwAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMJjABQAwmMAFADCYwAUAMNhFi5xUVRtJ/pjkd9NdD3f3F7YdczLJtUleTHJTd589l0IBANbVQoEryeVJHunu9+/0y6o6kuSdSa5K8qYk9ya5csHvAgBYa4sOKV6R5LKq+klV/aCqatvvjyY5092b3f1Ukouq6tA5VQoAsKZ27eGqquuTfGbb7hNJvtLd362qo0nuyit7sA4meXZm++9JXp/kz3O+auPQoUv3VDSrSfutL2233rTf+tJ2/z92DVzdfSrJqdl9VXVJJnOz0t0PVNVlVbXR3ZvTQ55LMvtXdGmSv56fkgEA1suiQ4onk9yUJFV1OMlTM2ErSR5McryqDlTVm5Mc6O6/nFupAADradFJ819NcldVbT2F+IkkqapbktzT3Wer6v4kD2cS6k6ch1oBANbSxubm5u5HAQCwMC8+BQAYTOACABhM4AIAGGzRSfPnVVV9KMlHu/tj0+23J7k9kwn5Z7r7S8usj/n2stQTq6eqDiT5ZpLDSf6Z5FPd/cRyq2KvqurRJH+bbv6huz+5zHrYm6q6KsnN3X2sqt6S5M4km0keS3Kiu19aZn3Mt639jiT5Xl6+932ru7/zaucuPXBV1e1Jjif59czuO5J8JMnvk3y/qo509yPLqI89mbvUEyvrg0ku7u53TP/JuTXJB5ZcE3tQVRcnSXcfW3Ip7ENVfT7Jx5P8Y7rrtiRf7O6fVtUdmVx/p5dVH/Pt0H5HktzW3bfu5fxVGFJ8KMmNWxtVdTDJ67r7yem7ve5Lcs2yimNPdlvqidV0NMmPkqS7f5Hkrcsth304nOSSqjpTVT+eBmZW35NJPjyzfUWSn00//zDJuy94RezHTu13bVX9vKpOVdXcZQMuWOCqquur6rFtP1dOu99m301xMJM31W/ZWhaIFbBTOyZ5JpOlnt6V5MuZLPXE6juYl4ekkuTfVbX0Xm/25PkkX8tkdOCGJHdru9XX3fcm+dfMrtkVWtzrVtwO7Xc2yee6++pMRuROzjv/gl2gOy0R9CosC7TCFlzqidW0/Vo70N0vLqsY9uXxJE9Mr7HHq+rZJG9I8vRyy2KfZudrudetn9PdvdVmp5N8fd7BqzCk+Ard/VySF6rq8ulk7ONJ7l9yWcy321JPrKYHk7wv+e+DKr9Zbjnsw3WZzLlLVb0xk97KPy21IhbxaFUdm35+b9zr1s19VfW26edrkvxq3sGr2gV9Q5K7k7wmk6cUf7nkephvx6WeWHmnk7ynqh5KspHEU27r41SSO6vqgUymZFynd3ItfTbJt6vqtUl+m+SeJdfD/tyY5BtV9UImU2s+Pe9gS/sAAAy2ckOKAAD/awQuAIDBBC4AgMEELgCAwQQuAIDBBC4AgMEELgCAwf4DPJfaTN4/hqYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num4, mu4, var4 = nums[3], true_Mu[3], true_Var[3]\n",
    "X4 = np.random.multivariate_normal(mu4, var[3], num4)\n",
    "plot_cluster(X4, [9, 4.5], [1, 3], 'c', 25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后来看一下汇总图"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![clusters](../img/clusters.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [变量初始化](#content)\n",
    "首先要对GMM模型参数以及隐变量进行初始化。通常可以用一些固定的值或者随机值。\n",
    "\n",
    "- n_clusters是GMM模型中聚类的个数，和K-Means一样我们需要提前确定。这里我们除去倾斜的蓝色数据，所以聚类个数为3。\n",
    "\n",
    "- n_points是样本点的个数。\n",
    "\n",
    "- Mu是每个高斯分布的均值。\n",
    "\n",
    "- Var是每个高斯分布的方差，为了过程简便，我们这里假设协方差矩阵都是对角阵。\n",
    "\n",
    "- W是上面提到的隐变量，也就是每个样本属于每一簇的概率，在初始时，我们可以认为每个样本属于某一簇的概率都是1/3。\n",
    "\n",
    "- Pi是每一簇的比重，可以根据W求得，在初始时，Pi = [1/3, 1/3, 1/3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**注：如何确定GMM中的聚类个数**\n",
    "\n",
    "- 方法一：BIC\n",
    "\n",
    "$$\n",
    "BIC=−2log(L)+klog(n)\n",
    "$$\n",
    "\n",
    "L是likelihood，k是component的个数，n是样本的个数。\n",
    "\n",
    "- 方法二：cross validation\n",
    "\n",
    "另一个方法是根据split test的结果（或者说cross validation的结果），先用训练集得到GMM的参数，然后再在测试集上计算log-likelihood。两者明显分叉的地方就是component个数的最佳候选。\n",
    "\n",
    "![](../img/clusters-likehood.png)\n",
    "\n",
    "参考：[https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html#sphx-glr-auto-examples-mixture-plot-gmm-selection-py](https://scikit-learn.org/stable/auto_examples/mixture/plot_gmm_selection.html#sphx-glr-auto-examples-mixture-plot-gmm-selection-py)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_clusters = 3\n",
    "n_points = len(X)\n",
    "Mu = [[0, -1], [6, 0], [0, 9]]\n",
    "Var = [[1, 1], [1, 1], [1, 1]]\n",
    "Pi = [1 / n_clusters] * 3\n",
    "W = np.ones((n_points, n_clusters)) / n_clusters \n",
    "Pi = W.sum(axis=0) / W.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [E步骤](#content)\n",
    "\n",
    "$$\n",
    "\\left.Q_{i}\\left(z^{(i)}\\right)=P\\left(z^{(i)} | x^{(i)}, \\theta^{j}\\right)\\right)\n",
    "$$\n",
    "\n",
    "E步骤中，我们的主要目的是更新W。第i个变量属于第m簇的概率为(这里的$\\pi_{m}$就是隐变量，我们最开始设置其为1/3，也就代表在当前的均值和方差的情况下，我们隐变量$\\pi_{m}的先验概率为1/3$)：\n",
    "$$\n",
    "W_{i, m}=\\frac{\\pi_{j} P\\left(X_{i} | \\mu_{m}, \\operatorname{var}_{m}\\right)}{\\sum_{j=1}^{3} \\pi_{j} P\\left(X_{i} | \\mu_{j}, \\operatorname{var}_{j}\\right)}\n",
    "$$\n",
    "根据W，我们就可以更新每一簇的占比$\\pi_{m}$，\n",
    "$$\n",
    "\\pi_{m}=\\frac{\\sum_{i=1}^{n} W_{i, m}}{\\sum_{j=1}^{k} \\sum_{i=1}^{n} W_{i, j}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def update_W(X, Mu, Var, Pi):\n",
    "    n_points, n_clusters = len(X), len(Pi)\n",
    "    pdfs = np.zeros(((n_points, n_clusters)))\n",
    "    for i in range(n_clusters):\n",
    "        # multivariate_normal.pdf：多元正态分布的概率密度函数\n",
    "        pdfs[:, i] = Pi[i] * multivariate_normal.pdf(X, Mu[i], np.diag(Var[i]))\n",
    "    W = pdfs / pdfs.sum(axis=1).reshape(-1, 1)\n",
    "    return W\n",
    "\n",
    "\n",
    "def update_Pi(W):\n",
    "    Pi = W.sum(axis=0) / W.sum()\n",
    "    return Pi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下是计算对数似然函数的logLH以及用来可视化数据的plot_clusters。\n",
    "\n",
    "$$\n",
    "L\\left(\\theta, \\theta^{\\prime}\\right)=\\sum_{i=1}^{m} \\sum_{z^{(i)}} Q_{i}\\left(z^{(i)}\\right) \\log P\\left(x^{(i)}, z^{(i)} ; \\theta\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def logLH(X, Pi, Mu, Var):\n",
    "    n_points, n_clusters = len(X), len(Pi)\n",
    "    pdfs = np.zeros(((n_points, n_clusters)))\n",
    "    for i in range(n_clusters):\n",
    "        pdfs[:, i] = Pi[i] * multivariate_normal.pdf(X, Mu[i], np.diag(Var[i]))\n",
    "    return np.mean(np.log(pdfs.sum(axis=1)))\n",
    "\n",
    "\n",
    "def plot_clusters(X, Mu, Var, Mu_true=None, Var_true=None):\n",
    "    colors = ['b', 'g', 'r']\n",
    "    n_clusters = len(Mu)\n",
    "    plt.figure(figsize=(10, 8))\n",
    "    plt.axis([-10, 15, -5, 15])\n",
    "    plt.scatter(X[:, 0], X[:, 1], s=5)\n",
    "    ax = plt.gca()\n",
    "    for i in range(n_clusters):\n",
    "        plot_args = {'fc': 'None', 'lw': 2, 'edgecolor': colors[i], 'ls': ':'}\n",
    "        ellipse = Ellipse(Mu[i], 3 * Var[i][0], 3 * Var[i][1], **plot_args)\n",
    "        ax.add_patch(ellipse)\n",
    "    if (Mu_true is not None) & (Var_true is not None):\n",
    "        for i in range(n_clusters):\n",
    "            plot_args = {'fc': 'None', 'lw': 2, 'edgecolor': colors[i], 'alpha': 0.5}\n",
    "            ellipse = Ellipse(Mu_true[i], 3 * Var_true[i][0], 3 * Var_true[i][1], **plot_args)\n",
    "            ax.add_patch(ellipse)         \n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [M步骤](#content)\n",
    "\n",
    "M步骤中，我们需要根据上面一步得到的W来更新均值Mu和方差Var。 Mu和Var是以W的权重的样本X的均值和方差。\n",
    "\n",
    "因为这里的数据是二维的，第m簇的第k个分量的均值，\n",
    "\n",
    "$$\n",
    "\\mu_{m, k}=\\frac{\\sum_{i=1}^{n} W_{i, m} X_{i, k}}{\\sum_{i=1}^{n} W_{i, m}}\n",
    "$$\n",
    "\n",
    "第m簇的第k个分量的方差，\n",
    "\n",
    "$$\n",
    "\\operatorname{var}_{m, k}=\\frac{\\sum_{i=1}^{n} W_{i, m}\\left(X_{i, k}-\\mu_{m, k}\\right)^{2}}{\\sum_{i=1}^{n} W_{i, m}}\n",
    "$$\n",
    "\n",
    "以上迭代公式写成如下函数update_Mu和update_Var。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def update_Mu(X, W):\n",
    "    n_clusters = W.shape[1]\n",
    "    Mu = np.zeros((n_clusters, 2))\n",
    "    for i in range(n_clusters):\n",
    "        Mu[i] = np.average(X, axis=0, weights=W[:, i])\n",
    "    return Mu\n",
    "\n",
    "def update_Var(X, Mu, W):\n",
    "    n_clusters = W.shape[1]\n",
    "    Var = np.zeros((n_clusters, 2))\n",
    "    for i in range(n_clusters):\n",
    "        Var[i] = np.average((X - Mu[i]) ** 2, axis=0, weights=W[:, i])\n",
    "    return Var"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [迭代求解](#content)\n",
    "\n",
    "下面我们进行迭代求解。\n",
    "\n",
    "图中实线是真实的高斯分布，虚线是我们估计出的高斯分布。可以看出，经过5次迭代之后，两者几乎完全重合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loglh = []\n",
    "for i in range(5):\n",
    "    plot_clusters(X, Mu, Var, [mu1, mu2, mu3], [var1, var2, var3])\n",
    "    loglh.append(logLH(X, Pi, Mu, Var))\n",
    "    W = update_W(X, Mu, Var, Pi)\n",
    "    Pi = update_Pi(W)\n",
    "    Mu = update_Mu(X, W)\n",
    "    print('log-likehood:%.3f'%loglh[-1])\n",
    "    Var = update_Var(X, Mu, W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每次迭代的log-likelihood如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "log-likelihood:-8.163\n",
    "log-likelihood:-4.701\n",
    "log-likelihood:-4.698\n",
    "log-likelihood:-4.697\n",
    "log-likelihood:-4.697"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](../img/em1.png)\n",
    "![](../img/em2.png)\n",
    "![](../img/em3.png)\n",
    "![](../img/em4.png)\n",
    "![](../img/em5.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### [完整代码](#content)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.patches import Ellipse\n",
    "from scipy.stats import multivariate_normal\n",
    "plt.style.use('seaborn')\n",
    "\n",
    "def generate_X(true_Mu, true_Var):\n",
    "    '''\n",
    "    生成三个高斯分布数据\n",
    "    :param true_Mu: 均值\n",
    "    :param true_Var: 方差\n",
    "    :return:\n",
    "    '''\n",
    "    # 第一簇的数据\n",
    "    num1, mu1, var1 = 400, true_Mu[0], true_Var[0]\n",
    "    X1 = np.random.multivariate_normal(mu1, np.diag(var1), num1)\n",
    "    # 第二簇的数据\n",
    "    num2, mu2, var2 = 600, true_Mu[1], true_Var[1]\n",
    "    X2 = np.random.multivariate_normal(mu2, np.diag(var2), num2)\n",
    "    # 第三簇的数据\n",
    "    num3, mu3, var3 = 1000, true_Mu[2], true_Var[2]\n",
    "    X3 = np.random.multivariate_normal(mu3, np.diag(var3), num3)\n",
    "    # 合并在一起\n",
    "    X = np.vstack((X1, X2, X3))\n",
    "    # 显示数据\n",
    "    plt.figure(figsize=(10, 8))\n",
    "    plt.axis([-10, 15, -5, 15])\n",
    "    plt.scatter(X1[:, 0], X1[:, 1], s=5)\n",
    "    plt.scatter(X2[:, 0], X2[:, 1], s=5)\n",
    "    plt.scatter(X3[:, 0], X3[:, 1], s=5)\n",
    "    plt.show()\n",
    "    return X\n",
    "\n",
    "# E步骤更新W，也就是第i个变量属于第m簇的概率\n",
    "# 更新W\n",
    "def update_W(X, Mu, Var, Pi):\n",
    "    n_points, n_clusters = len(X), len(Pi)\n",
    "    pdfs = np.zeros(((n_points, n_clusters)))\n",
    "    for i in range(n_clusters):\n",
    "        # multivariate_normal.pdf：多元正态分布的概率密度函数\n",
    "        pdfs[:, i] = Pi[i] * multivariate_normal.pdf(X, Mu[i], np.diag(Var[i]))\n",
    "    W = pdfs / pdfs.sum(axis=1).reshape(-1, 1)\n",
    "    return W\n",
    "\n",
    "# 根据更新的W，更新每一簇的占比\n",
    "# 更新pi\n",
    "def update_Pi(W):\n",
    "    Pi = W.sum(axis=0) / W.sum()\n",
    "    return Pi\n",
    "\n",
    "\n",
    "# 计算log似然函数\n",
    "def logLH(X, Pi, Mu, Var):\n",
    "    n_points, n_clusters = len(X), len(Pi)\n",
    "    pdfs = np.zeros(((n_points, n_clusters)))\n",
    "    for i in range(n_clusters):\n",
    "        pdfs[:, i] = Pi[i] * multivariate_normal.pdf(X, Mu[i], np.diag(Var[i]))\n",
    "    return np.mean(np.log(pdfs.sum(axis=1)))\n",
    "\n",
    "\n",
    "# 画出聚类图像\n",
    "def plot_clusters(X, Mu, Var, Mu_true=None, Var_true=None):\n",
    "    colors = ['b', 'g', 'r']\n",
    "    n_clusters = len(Mu)\n",
    "    plt.figure(figsize=(10, 8))\n",
    "    plt.axis([-10, 15, -5, 15])\n",
    "    plt.scatter(X[:, 0], X[:, 1], s=5)\n",
    "    ax = plt.gca()\n",
    "    for i in range(n_clusters):\n",
    "        plot_args = {'fc': 'None', 'lw': 2, 'edgecolor': colors[i], 'ls': ':'}\n",
    "        ellipse = Ellipse(Mu[i], 3 * Var[i][0], 3 * Var[i][1], **plot_args)\n",
    "        ax.add_patch(ellipse)\n",
    "    if (Mu_true is not None) & (Var_true is not None):\n",
    "        for i in range(n_clusters):\n",
    "            plot_args = {'fc': 'None', 'lw': 2, 'edgecolor': colors[i], 'alpha': 0.5}\n",
    "            ellipse = Ellipse(Mu_true[i], 3 * Var_true[i][0], 3 * Var_true[i][1], **plot_args)\n",
    "            ax.add_patch(ellipse)\n",
    "    plt.show()\n",
    "\n",
    "# M步根据更新的W和PI来跟新均值Mu与方差Var\n",
    "# 更新Mu\n",
    "def update_Mu(X, W):\n",
    "    n_clusters = W.shape[1]\n",
    "    Mu = np.zeros((n_clusters, 2))\n",
    "    for i in range(n_clusters):\n",
    "        Mu[i] = np.average(X, axis=0, weights=W[:, i])\n",
    "    return Mu\n",
    "\n",
    "\n",
    "# 更新Var\n",
    "def update_Var(X, Mu, W):\n",
    "    n_clusters = W.shape[1]\n",
    "    Var = np.zeros((n_clusters, 2))\n",
    "    for i in range(n_clusters):\n",
    "        Var[i] = np.average((X - Mu[i]) ** 2, axis=0, weights=W[:, i])\n",
    "    return Var\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    # 生成数据\n",
    "    true_Mu = [[0.5, 0.5], [5.5, 2.5], [1, 7]]\n",
    "    true_Var = [[1, 3], [2, 2], [6, 2]]\n",
    "    X = generate_X(true_Mu, true_Var)\n",
    "    # 初始化\n",
    "    n_clusters = 3 #聚类的个数\n",
    "    n_points = len(X)\n",
    "    Mu = [[0, -1], [6, 0], [0, 9]]\n",
    "    Var = [[1, 1], [1, 1], [1, 1]]\n",
    "    Pi = [1 / n_clusters] * 3\n",
    "    W = np.ones((n_points, n_clusters)) / n_clusters #隐变量\n",
    "    Pi = W.sum(axis=0) / W.sum() #每一簇的比重，可以根据W求得\n",
    "    # 迭代\n",
    "    loglh = []\n",
    "    for i in range(5):\n",
    "        plot_clusters(X, Mu, Var, true_Mu, true_Var)\n",
    "        loglh.append(logLH(X, Pi, Mu, Var))\n",
    "        W = update_W(X, Mu, Var, Pi)\n",
    "        Pi = update_Pi(W)\n",
    "        Mu = update_Mu(X, W)\n",
    "        print('log-likehood:%.3f'%loglh[-1])\n",
    "        Var = update_Var(X, Mu, W)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## [参考文献](#content)\n",
    "\n",
    "[1] 刘建平：[最大熵模型原理小结](https://www.cnblogs.com/pinard/p/6093948.html)\n",
    "\n",
    "[2] 忆臻: [一文搞懂极大似然估计](https://zhuanlan.zhihu.com/p/26614750)\n",
    "\n",
    "[3] 知行流浪：[极大似然估计详解](https://blog.csdn.net/zengxiantao1994/article/details/72787849)\n",
    "\n",
    "[4] 刘建平：[EM算法原理总结](https://www.cnblogs.com/pinard/p/6912636.html)\n",
    "\n",
    "[5] v_JULY_v：[如何通俗理解EM算法](https://blog.csdn.net/v_JULY_v/article/details/81708386)\n",
    "\n",
    "[6] 林立民爱洗澡：[详解EM算法与混合高斯模型(Gaussian mixture model, GMM)](https://blog.csdn.net/lin_limin/article/details/81048411)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
